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ABSTRACT 

Designed to define the academic writing skills 
required cf beginning undergraduate and graduate students, a survey 
of needed academic writing skills was completed by faculty in 190 
academic departments at 34 AmGrican and Canadian universities with 
high foreign student enrollments. At the graduate level, six academic 
disciplines with relatively high numbers of non~native students were 
surveyed: business management, civil and electrical engineering, 
psychology, chemistry, and computer science. Undergraduate English 
departments v/ere chosen to document the skills needed by 
undergraduate students. The faculty members surveyed appear to view 
the written communicative competencies of their students 
predominantly from the perspective of sociolinguistic competence, 
placing considerably less emphasis on grammatical competence. 
Although some important common elements amon^ the different 
departments were reported, the survey data distinctly indicate that 
different disciplines do not uniformly agree on the writing task 
demands and on a single preferred modf. of discourse for evaluating 
enterhig undergraduate and graduate stv^dents. (Author/PN) 
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The Test of English as a Foreign Language (TOEFL) was developed in 1fl63 by 
a National Council on the Testing cf English as a Foreign Language^ which 
was formed through the cooperative effort of over thirty organizations^ public 
and private, that were concerned with testing the English proficiency of non 
native speakers of the language applying for admission to institutions in the 
United States, in 1965^ Educational Testing Service (ETS) and the College 
Board assumed joint responsibility for the program and in 1973 a cooperative 
arrangement for the operafion of the program was entered into by ETS^ the 
College Board, and the Graduate Record Examinations (GRE) Board. The 
membership of the College Board is ccmposed of schools, colleges^ school 
systemSi and educational associations; GRE Board members are associated 
with graduate education. 

ETS administers the TOEFL program under the general direction of a Policy 
Council that was established by^ and is affiliated with, the sponsoring organi* 
zations. Members of the Policy Council represent the College Board and the 
GRE Board and such institutions and agencies as graduate schools of 
busmesst funiot and community colfeges, nonprofit educaHonaf exchange 
agencies^ and agencies of the United States government. 



A continuing program of research related to TOEFL is carried out under the 
direction of the TOEFL Research Committee. Its six members include repre- 
sentatives of the Policy Council, the TOEFL Commiffee of Examiners^ and 
distinguished English*as*a*second*language specialists from the academic 
community. Currently the committee meets twice yearly to review and ap- 
prove proposals for test*related research and to set guidelines for the entire 
scope of Ihe TOEFL research program. Members of the Research Committee 
serve three-year terras at the invitation of the Policy Council; the chair of the 
committee serves on the Policy Council. 

Because the studies are specific to the test and the testing program* most of 
Ihe actual research ts conducted by ETS staff lather than by outside re* 
searchers. However, many projects require the cooperation of other institu* 
tfons. particularly those with programs in the teaching of English as a foreign 
0*: second language. Representatives of such programs who are interested in 
participating in or conducting TOEFL*related research are invited to contact 
the TOEFL program office. Local research may sometimes require access fo 
TOEFL data. In such cases, the program may provide this data following 
approval by the Research Committee. All TOEFL research projects must 
undergo appropriate ETS review to ascertain that the confidentiality of data 
will be;jrotected. 

Current (1981 82) members of theTOEFL Research Committee include the fol- 
iowir. 

G. Richard Tucker (chair) Cenfer ^or Applied Linguistics 
Louis A. Arena University of Delaware 

H. Douglas Brown University of Illinois at Urbana-Champagne 
Frances B. Hinofotis University of California at Los Angeles 
Diane Larsen- Free man The Experiment in International Living 
David S. Sparks University of Maryland 
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Abstract 



A survey of the academic writing skills needed by beginning under-* 
graduate and graduate students was conducted* Faculty in 190 academic 
departments at thirty^four U.S. and Canadian universities with high 
foreign student enrollments completed the questionnaire. At the graduate 
level » six academic disciplines with relatively high numbers of nonnative 
students were surveved: business management (MBA), civil engineering, 
electrical engineering, psychology, chemistry, and computer science. 
Undergraduate English departments were chosen to document the skills 
needed by undergraduate students. 

The major findings are summarized below. 

o Although writing skill was rated '^as important to success in 
graduate training, it was consistently rated as even more 
important to success after graduation. 

o Even disciplines with relatively light writing requirements 
(e.g., electrical engineering) reported that some writing 
is required of first-year students. 

o The writing skills perceived as most important varied across 
departments. 

o Faculty members reported that, in their evaluations of 
student writing, they rely more on discourse-level charac- ^ 
teristics than on word" or sent«nce-level characteristics. 

o Discourse- lev el writing skills of natives and nonnatives vere 
perceived as fairly similar, but significant differences 
between natives and nonnatives were reported for sentence- 
and word-level skills and for overall writing. 

o Among the ten writing sample topic types provided, preferred 
topic types differed across departmeats. 

Although some important common elements among the different depart- ^ 
mencs were reported, the survey data distinctly indicate that different 
disciplines do not uniformly agree on the writing task demanda and on a 
single preferred mode of discourse for evaluating entering undergraduate 
and graduate students. 
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Section I. Introduction and Background 



The Test of English as a Foreign Language (TOEFL)* was designed to 
assist academic institutions in determining whether applicants have 
attained sufficient proficiency in English to study at those institutions. 
\n important component of that general proficiency is the ability to 
write clearly and grammatically in English. A recent informal survey of 
professionals in the field of English as a second language (ESL) by Hale 
and Hinofotis (l98l » 1983) identified the measurement of productive skills 
(e.g.» speaking and writing) as potentially useful for preadmission 
testing and for making placement decisions. A report by Angelis (l982) 
reached the same general conclusion. Angelis surveyed graduate faculty 
members in engineering and business, the two fields that enroll the 
largest numbers of foreign students. He found that graduate faculty 
memoers in engineering ranked writing highest on the list of foreign 
student deficiencies; business faculty also listed the writing defici- 
encies of foreign students as a major concern. Furthermore, many 
respondents believed that TOEFL does a limited job of providing 
info^ation about productive skills such as writing. Most frequently, 
respondents felt that there may be little relationship between knowledge 
of grammar and actual writing skill. Therefore, it is important to 
determine the extent to which the current version of TOEFL is a valid 
indicator of the English writing skills required of applicants to under- 
graduate and graduate institutions in the United States and Canada* 
However, before a meaningful validation study of TOEFL can be conducted, 
the academic writing skills required of beginning undergraduates and 
graduate students must be defined. The purpose of this project was to 
provide that definition. 



Criticisms of Current Writing Assessment Measures 

Other standardised examinations and assessment procedures recently 
have dr&wxi criticism, typically from linguistics and English language 
educrvtors, with respect to their narrow definition of writing competence. 
Troyka (l982) , in fact, traces a decade of resolutions brought to the 
annual meetings of the Conference on Coll- ^e Composition and Communication 
(CCCC), a major affiliate of the National Council of Teachers of English 
(NCTE). Throughout the 1970s, this organization condemned multiple- 
choice measures of writing as narrowly focused and as gross distortions 
of writing competence. Expressing a preference for tests that require 
direct measures, or writing samples, the group now attanpts to recommend 
more valid alternatives to the indirect measures to '^ich they object. 
Lloyd**Jones ( 1982) criticizes both indirect and direct measures of 
writing that are currently in use, warning that they may understate 
the competence of many candidates who might otherwise succeed. Rather 
than being abolished, these tests must be interpreted in light of their 
limitations. Although ^^nriting samples most nearly approximate real 
discourse, numerous problems also are associated with them; Lloyd-Jones 
claims, **A writing sample is not real writing** (l982, p. 3). He recog- 
nizes, however, that objective (indirect) measures are based on limited. 
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discrete elements! of language. The evaluation of separate elements of 
wilting skills » h€ 'believes » is not equivalent to the evaluation of actual 
discourse, since ^ood writing consists of a blend of skills, not of an 
additive sum of the'se skills viewed Separately. 

; y 

Odell (1981),: a noted spokesman in the field of writing instruction 
and assessment, voices similar views. He defines competence in writing as 
"... the ability to discover what one wishes to say and to convey one*d 
message through language, syntax and content that are appropriate for 
one's audience and purpose" (p. 103). In contract to the more narrowly 
constrained measures of writing ability presently available, Odell 
cites the work o| Moffett (1968), Kinneavy (l97l), and others, who have 
clarified the great diversity of writing tasks in the "real" world. 
Although all the^orists do not agree on the specific categorization of 
tasks, they generally agree that: 
f 

1, Writing :includes many modes, with diverse purposes and audiences. 

2. These different mod»s, purposes, and audiencAS require different 

organisational strategies of written production. 

I 

Thus, Odell likewise criticizes current indirect measures of writing 
ability for their^' undue emphasis on error recognition, and for their 
failure to reflect the skills that are needed to generate good writing. 
With respect; to ^is definition of writing competence, one that is pre** 
dominantly accefyted by others in the field, Odell also highlights the 
shortcomings of direct measures of writing as they are prevalently 
Used: the ^aiVure to specify purpose and audience, snd the varying 
perception^; of /judges, who are apt to use different standards and attend 
to different aspects of writing in their evaluations of writing samples. 
Odell emphjasizes the need to analyze writing sample topics from the 
standpoint of the skills aiid tasks that are required, e.g., chronological 
or analytical development, drawing from different sources, recognizing 
personal assumptions or the assumptions of others. 

This prevalent definition of writing competence, with the recogni- 
tion that written production may call for diverse modes, purposes, and 
audiences that require different organizational structures, was recently 
examined in a study conducted by Quellmalz, Capell, and Chou (1982). They 
attempted to compare the writing competency profiles derived by tests 
differing in. discourse mode and response mode. Measures of writing can 
demand different discourse modes, such as tiarrative or expository, which 
may tap different cognitive skills, hwce different levels of performance 
within the modes. * In addition, measures of writing can asseas skills 
either^ directly (production) or .ndirectly (recognition), thus Imposing 
task features that elicit different strstegies and possibly yield 
different evaluations of writing competence. These researchers stated 
that, "In practice, many current writing assessment progrsms fail to 
consider the validity of test dd^«« ehat do not distinguish among the 
demanda of writing tasks and between the requirements of production and 
selection. At heart, the issue is one of construct vslidity: do these 
alternative task and processing variables measure the same thing? Our 
results indicate thst this is not the case" (p. 255). 
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Quellmalz and associates subjected their data on writing measures to 
several analyses, confirming that levels of performance vary on tasks with 
different writing purposes, thus questioning . the assumption that *'a good 
writer is a good writer" (p« 255)* These results emphasize the importance 
of clarifying the demands that are required by different, specific writing 
tasks; with regard to their comparison of direct and indirect measures, 
however, the results were not as definitive. Quellmalz and associates 
noted that response mode effects were present in varying degrees; in 
particular, organization was most influenced by the response requirements 
of the task. Vith respect to their comparison results concerning 
discourse modes, however, they concluded that the time savings obtained 
with more indirect measures of writing do not offset the loss of .distinc- 
tive information obtained through direct measures. 

The foregoing views of writing competence, especially the recognition 
of the specific task demands elicited by particular writing tasks, reflect 
the current orientation in the field of linguistics toward the approach 
known as functionally based comnunicative competence. 

Functionally Based Communicative Competency 

Linguists who have investigated the dimensions of second language 
teaching and testing (Canale & Swain, 1979; Munby, 1981; Walz, 1982) 
stress a functionally based communicative approach, as opposed to a 
grammatically based approach. One justification for tnis emphasis i^ the 
face validity of the materials and syllabus on which second language 
learning is based. From the standpoint of the learner, Canale and Swain 
provide two ^'subjective reasons" for theli' point of view: (l) the more 
positive consequences for learner motivation resulting from less emphasis 
on "communicative incompetence" and more emphasis on the communicative 
purpose of language and (2) the '*more natural integration of knowledge 
of the second language culture,- knowledge of the second language, and 
knowledge of language in general'* (p. ^0). Such functional approaches to 
second language learners underscore the previously mentioned discrepancies 
between knowledge of grammar and actual written production in present 
measures of language skills. With regard to the testing of communication 
In a second language, Canale and Swain, among others, distinguish between 
communicative competence and performance: 

The fundamental theoretical distinction that we have 
accepted between communicative competence and per- 
formance suggests that communicative testing must be 
devoted not only to what the learner knows about the 
second language and about how to use it (competence) 
but also to what extent t^e learner is able to 
actually demonstrate this knowledge in a meaningful 
communicative situation (performance). It has bee.a 
emphasized quite frequently (e.g., by J. B. Carroll, 
1961; Clark, 1972; Jones, ,1977; Morrow, 1977; and in 
press. Oiler, 1976) that pencil-and-Paper tests now ia 
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use do not necessarily give valid Indication of second 
language learners' skills In performing In actual 
com!dunlcatlve situations* Our theoretical framework 
su^Sgests the boundaries and contents of coiFuiiunlcatlve 
crjmpetence that are necessary and Important for this 
rype of performance* Ue think that It Is Important 
to empirically study the extent to which competence-' 
oriented tests are valid Indicators of learners* 
success In handling actual performance* (pp* 62-63) 



Within the Canale and Swain framework, we assume that the student^s 
communicative needs In genuine communicative eltuatlous should be 
specified with respect to the following: 

o Grammatical competence — level of graamatlcal accuracy 
required; knowledge of lexical Items and rules of 
morphology , syntax , sentence-grammar semantics, and 
phonology* 

o Soclollngulstlc competence — soclocultural rules of use 
and rules of discourse* This dimension of competence 
Includes two sets of rules: (1) appropriateness of 
written utterances within a given soclocultural context 
(contextual factors such as topic, role of participant, 
setting, and norms of Interaction), and (2) appropriate 
attitude or style conveyed by a particular grammatical 
form within a given soclocultural context* 

o Strategic competence — gt^matlcal and soclollngulstlc 
strategies to be used when there Is a breakdown in 
other competencies* These "coping strategies" (Swain, 
1978) may be helpful to students at the beginning 
stages of second language learning, and the need for 
these competencies may cliange as a function of age and 
second language proficiency* 



Wrltlqg Task Demands within Functional Contexts 

The previous arguments stress the Importance of more precisely 
Identifying written comminlcatlon^performance that Is demanded within 
functional contexts in order to a/rlve at a more accurate description 
of the kinds of writing tasks that are required of undergraduate and 
graduate second language students* Recently a number of researchers have 
attempted to Identify some of these tasks, with a focus on Improving 
Instruction by more directly addressj^ng students* communicative needs 
througu Instruction* Kroll (1979) surveyed a small number of students In 
.reshmau English courses for International students and In comparable 
courses for native speakers of English at the University of Southern 
California* Her results Indicated that the past writing experiences and 
current writing needs of foreign and American students were predictable 
and similar, justifying the requirement that foreign students take the 
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English writing course. These foreign students had past experience with 
written English and also expected to use written English in the future. 
The writing task demands that Kroll identified resulted in her recommen- 
dation that students be provided opportunities to gain experience with 
modes of discourse they will be called upon to use; e.g., the personal 
essay was found to be less important than were business letters of 
persuasion, reports, and other writing tasks. 

Surveying the academic needs for advanced ESL stt?uents, also at 
the University of Southern California, Ostler (1980), reported a clear 
distinction between the academic skills needed by graduate and under- 
graduate student^s; some of these skills were specific to major fields. 
Johns (1981) surveyed randomly selected faculty members from all depart- 
ments at San Diego State University to determine which academic skills 
(reading, writing, speaking, listening) were most essential to the success 
of nonnative speakers of English in their university classes. All 
departments rated the receptive skills (reading and listening) as most 
important, suggesting that writing was a skill that should be taught as 
secondary to the receptive skills. Although most of the faculty agreed 
that general English was of more benefit to the students, the engineering 
faculty preferred special^^purpose English over general English. 

In contrast to the Johns result with engineering departments. 
Weaver (1982) found that • • faculty from many disciplines expressed 
similar values about writing..." (p. 12). Her approach involved studying 
the standards of writing competency used by faculty readers of writing 
samples. In general, these faculty members defined competency in terms of 
careful and logical organization of ideas, placing less emphasis on 
errors; the readers, however, tended to seek errors in poorly developed 
essays and to ignore errors In well-organized prose. Freedman (1979) used 
a similar approach to studying features of writing that are valued in 
compositions by comparing the holistic ratings of trained evaluators. 
College students* expository essays were rewritten to be stronger or 
weaker in four a teas: content, organization, sentence structure, and 
mechanics. Ratings were most affected by content and organization, with 
smaller effects attributed to mechanics and sentence structure. However, 
Freedman obtained a significant interaction effect of mechanics and 
sentence structure with organization. 



Field-Specific Writing Task Demands 

Other researchers have focused their research concerned with acadasic 
writing task demands on field-specific requirements, with emphasis on 
English for special purposes (ESP). For example. West and Byrd (l982), 
surveyed twenty-five engineering faculty members at the University of 
Florida to identify the kinds of writing assigned to graduate students 
during one academic year (1979-80). They determined that examinations, 
quantitative problems, and reports (research and technical) were required 
most frequently ; homework and papers , less frequently ; and progress 
reports and proposals, least frequently. West (l982) also surveyed 
thirty-three engineering faculty members during the same year, asking them 
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to rate American and foreign students on eight writing dimensions. Ttiese 
faculty ranked all foreign graduate student writing dimensions lower than 
the same writing dimensions of American students> except for quality of 
content. Making palrwlse comparisons on the eight dimensions of foreign 
student writing, West ordered the dimensions from weakest to strongest> as 
follQWs (p. 7) : 

1. Correctness of punctuation 

2. Quality of sentence structure 

3. Vocabulary size 

A. Correctness of vocabulary usage 

5. Quality of paragraph organization 

6. Quality of overall paper organization 

7. Quality of content 

8. Overall writing ability 

His results Indicated that ESL graduate engineering students needed help 
with sentence-level wrl!:ing skills* 

In another study that typifies research In writing for academic 
purposes> Johns (1980) focused on the cohesive elements In written 
business discourse. Three types of written discourse were coded for 
cohesive elements. Johns was able to Identify **const£llatlons" as 
cohesive elements In the types of dlscotirse but concluded that generali- 
zations about cohesive features In broad classes of discourse could not 
be made. Lexical cohesion, for example, was the most common category 
In all three discourse types but varied considerably among them. In 
addition, Johns noted that other features of written discourse In English 
of business and economics (EBE) need to be distinguished. Hill and 
associates ( 1982), stressing the academic need for ESL students to learn 
to write experimental-research papers, have outlined an instructional 
approach that similarly aims at functional discourse. Pointing to the 
growing interest in ESP and in English for academic purposes (EAP), these 
researchers identified experimental-research papers as important to 
academic and professional success in the sciences and social sciences. 
Their instructional method explicates the rhetorical divisions of this 
type of paper> to enable ESL students to understand the principles of 
organization that are required. 

As another example of the current orientation toward ESP> the Insti- 
tute of Electrical and Electronic Engineers devoted an entire publication 
(the March, 12, 1982, IEEE Transactions on Professional Communlcatlcn ) to 
the subject of oraJ and written communication within the engineering 
profession (Landesman, 1982). Some of the articles reflect the views of 
functional communicative competency; e.g., one article, by J. Maynard, 



explains how brtter procedural manuals can be written with the user and 
purpose In inlnd^ with stress on being "function-oriented" rather than 
"software-oriented/* 



Contrastlve Rhetoric ^ 

An'other area of research that has explored the academic task demands 
on nonnatlve speakers of English has been tamed *'coiitrastlve rhetoric.'* 
In thlf; area, rhetorical patterns across cultures are Identified and 
compared (Kaple »^72, 1976, 1977, 1982). The results of studies of 
contrastlve rhetoilc provide somewhat mixed evidence, some rejecting 
and others supporting the underlying assumption that the structural 
differences between the native language and the foreign language may 
Interfere with the learning of the foreign language. We reviewed several 
representative papers In this area In order to take cultural differences 
into account. 

Buckingham (1979) describes the operational levels of Instruction In 
the teaching of composition, providing a description of skills at each 
level (l-lll). Particularly at Level 111, the ^'advanced" level, students 
should be prepared to write ror a variety of communicative purposes, and 
soclollnguls tic factors of language use In the academic setting should 
be emphasized. Buckingham circumscribes this level of communicative 
competence as a point In which students learn flexibility Irt reaching a 
specific audience to Influence the "mental or physical behaviors In the 
reader" (p. 246). He distinguishes between the "conceptual paragraph" 
and the physical, or mechanical, aspects of paragraphing; ESL students 
need to recognize that the elements of a paragraph become united by a 
s Ingle theme through the logical sequencing of Ideas. Significantly , 
nonnatlve speakers of English tend to organize their writing In the manner 
In which they organize writing In their first language, or In "un-English 
discourse" (p. 250). Thus cultural differences In logical development 
emerge* In English, for Instance,, we employ logical proof, culturally 
defined levels of formality, and cultural referents; in other languageii, 
different syntactic choices and kinds of proof that are selected and 
valued by particular cultures are more predominant. Similarly, Dubln and 
Olshtaln ( 1980) stress the differences in rhetorical patterns from one 
culture to another, patterns that Influence cohesion and communicative 
purposes V They emphasize the importance of reading in English for ESL 
students of reading in English, a task that will expose them to the 
elements of English prose style that can be transferred to written 
coiq)Osltlon. 

Carpenter and Hunter (198.) studied approximately six hundred 
compositions of writers whose native languages were Engtlsh, Semitic, 
Oriental, Romance, or Slavic. They noted significant differences between 
the continuation of expository paragraphs among writers with different 
native languages. Influenced by the differences in patterns of logic 
Imposed on the order of ideas in the paragraphs. These differances, 
they suggest, resul from systematic differences in cultural styles of 
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thinking. In light of this evidence, Carpenter and Hunter recommend 
functional exercises to Improve coher3nce In ESL writing; they emphasize 
the need for a functional approach because the understanding of communi- 
cative functions contributes to overall organization and coherence. 
Llndstrom (1981) also underscores the pedagogical Implications of contras- 
tive analysis^ viewed from the standpoint of error gravity, or the extent 
to which errors Influence communication. The judgments of educators and 
,evaluators of writing tasks may be Influenced by their idiosyncratic 
experiences and language backgrounds; the perception of errors varies 
with the situation. In academic writing, when errors substantially 
affect rhetorical patterns, logical and coherent comunlcatlon is affected 
as well. 

According to Pearson (1981), the academic slU^Hs needed by ESL 
students at entry level in degree programs a^re further hampered by the 
students* problems with handling the concepts that underlie these skills. 
Many of these concepts reflect a culture-bound way of thinking that is 
unfamiliar and illogical to foreign students; they require considerable 
practice to be grasped, even- by American students. Pearson recomm&nds 
that the concepts that cut across the communication skills (reading, 
writing^ listening, speaking) should be transmitted to stttdents beginning 
in the lowest- level ESL classes. Concepts such as paraphrasing and 
s*^ .marlzlng, the general-specific dimension, and the relevance-irrelevance 
dimension should be presented to ESL students within a functional 
communicative approach, rather than through the traditional order of 
presentation of discrete basic skills. ESL specialists (Blenton, 1982; 
Taylor, 1982) suggest that the linguistic and cognitive approaches to 
academic learning situations need to be identified so that ESL teachers 
can more effectively help students meet communication expectations. In 
particular, Blenton notes that ESL students need to learn to communicate 
in modes that will meet future academic requirements. Unfortunately, ESL 
students who receive early instruction in personalized, narratlve"based 
writing may not be aware that these narrative writing tasks cannot be 
applied to all academic writing; thus their writing falls to meet the 
academic expectations of ImpersonaJ, formal exposition. 

In juxtaposition to the contra stive rhetoric position, some 
researchers suggest that implicit "unlversals*' in scientific and technical 
languages, or common rhetorical or grammatical processes, may (or may 
not) exist. Sellnker and associates ( 1978), for example, investigated 
rhetorical function shifts in English for science and technology (EST) 
discourse within a single paragraph. Their results indicated that EST 
students typically confuse rhetorical levels in the elementary stage of 
second language learning, and that rhetorical process development is 
only one fc^m of EST paragraph development. They recotmnend research to 
investigate the rhetorical and grammatical processes of International 
scientific and technical languages, acknowledging the poissibility that 
"...universal mode5 of thought and practice...'* (Wlddowson, 1974, p. 40) 
may be shared in EST. Alternatively » these possibly common processes may 
be explained by the fact that the scientific and rhetorical information of 
the world is coded in English. 
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Validation of the TOEFL Examination 



The* foregoing review of the literature, In addition to many other 
sources we consulted, enabled us to determine the more significant dimen- 
sions of the functional communicative demands In writing that might 
feasibly be imposed on entry-level nonnatlve speakers of English. These 
writing task demaQds can be described as general academic skill and 
assignment requl rements > and as field-specific competencies, placed within 
the situational contexts that Include academic settings and the culturally 
bused expectations and perceptions of those who evaluace student writing. 
As Cronbach (1971) has noted, It Is not really tests that are validated, 
but rather interpretations of data from tests used In specific contexts* 
Given the Intended purpose of TOEFL as a test designed to measure the 
rc.uiintiss of foreign students to respond to academic instruction In 
Liii;lish, It would not be reasonable to validate TOEFL against all conceiv- 
able writing tasks but only against those that beginning undergraduate and 
rriJdiiaLe students are likely to encounter as part of their college and 
university courtses. 

Although there may be s^me danger In attempting to Identify criteria 
for written expression in too many functional contexts, It also may be 
Inadvisable to define the contexts too narrowly. Previous TOEFL valida- 
tion studies Ce.g.> Pitcher & Ra , 1967; Echternacht, 1970) have used 
limited definitions of the desired criterion performance (e.g., a brief 
essay written on a general topic under a strict time limit and evaluated 
holist ically ) . The brief essay written In class niay be an important 
form ^f academic writing, but there are also longer terra papers, lab 
reports > and so forth. 

Furthermore> this earlier research correlated ratings of essays with 
subtests on the "old" TOEFL (a version with five subscores that was In use 
prior to 1976); since Plke*5 study (1979), and on the basis of his data, 
the writing ability and English structure subtests have been combined. 
Pike also concluded that the relationships found between essay ratings and 
tue writln^i ability section of the test provided little support for 
replacing this section with a writing sample. Referring to the writing 
research of Godshalk ar*J associates (1966), which Indicated that the 
ratings of essays vary from topic to topic. Pike used four different 
topics in his study. However, the writing demands elicited by these 
topics might not serve as satisfactory criteria today, when viewed 
from the standpoint of obtaining valid samples of functional writing 
competency . Two of the four topic s , using pic tures as stimuli, each 
required a sequentl/ I description of events; these would tend to elicit 
writin^^ skills that have been constrained by the topic and might not 
noCGSsarlly provide the student with sufficient opportunity to demonstrate 
writing ability. These topics. In providing a predetermined framework, 
might not tap the student's ability to organize Ideas, a skill that Is not 
measured by the discrete objective Items on TOEh^L; It thus follows that 
they would not be expected to supply Information that would supplement the 
objective measures. The other two topics, the writing of a dialogue and 
the comparison of the advantages of city and country life, required the 
student to incorporate certain words In the writing sample, a constraint 
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th a t seeiss to c reate unreasonable demands ; the topics al so may have 
Interposed cultural demands that prevented some students from demonstra- 
ting their writing skills adequately* Finally, the time limit allotted to 
each sample (ten m?,nutes) may also have restricted the possibility of 
organizing cohesive discourse. Foreign students, in particular, should be 
allowed enough time to process their ideas from one language to another 
(Lay, 1982). The Pike study provided sound data to support the construct 
validity of TOEFL. Since that time, we have acquired additional knowledge 
about the design of writing samples, knowledge that should enable us to 
structure topics that will elicit pe rf oiisance of the kinds of skills 
they are likely to measure. Moreover, since Pikers study involved the 
relationships between ratings of writing samples and the old form of 
TOEFL, these relationships should be reexaiiiined. 

The primary objective of this project was to identify and describe 
operationally the expectations of writing competence required of nonnative 
speakers at the beginning of their e.^ucational experiences in institutions 
of higher education in the United States and Canada. The information we 
gathered took into account the various factors that should be considered 
in defining communicative competence in writing — the functional task 
demands for which students are expected to be prepared, as well as the 
perceptions, sometimes culturally influenced, of those who evaluate them* 
The informal interviews and literature review provided the basis for the 
design of a survey instrument that incorporated the full range of expecta- 
tions of writing competence* The writing task demands , features of 
writing tasks, and types of writing sample topics were expressed in 
terminology that would communicate clearly to individuals in various 
disciplines. S^tbsequently , a representative sample of departments within 
institutions responded to the questionnaire, in order to provide a basis 
for describing the domain of writing competencies expected of entering 
native and nonnative students. This report presents a summary of the 
results of the investigation. With this data base, we will be^able to 
design more valid direct measures of writing ability that could be 
related to scores on TOLFL in the future. 
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Sectlon !!• Method 



This section describes the questionnaire development strategy ^ 
questionnaire content, development of topic types, sampling plan, and data 
collection strategies. 



Questionnaire Development Strategy 

Three primary sources were used to develop the questionnaire* First, 
the literature review (see preceding section) suggested several general 
areas of concern that should be addressed and indicated specific questions 
that should be asked* Second, the study's advisory committee* met twice 
and made a number of useful suggestions* Third, a series of interviews 
was conducted with faculty at nearby institutions* 

Interviews * The interview portion of the study was conducted during 
June and July 1982* Interviews were held with about thirty faculty 
members from the following universities: Columbia University, the Univer* 
sity of Delaware^ New York University, tie University of Pennsylvania, 
Princeton University, and Rutgers University* These institutions were 
selected because they had substantial numbers of foreign students and 
were within easy travel distance from the £TS Princeton office* Most 
interviewees were from engineering departments, business departments, or 
English language institutes* The interviews were relatively unstructured 
but focused on four major points: (1) a description of the kinds of 
writing tasks assigned td first-year students, (2) a description of the 
standards used to evaluate student writing, (3) identification of the 
kinds of writing problems that appear to be especially severe for foi^ign 
students, and (4) a determination of the kind of writing task that would 
be most useful on an admissions test* 



Because interview responses were used primarily to determine 
important issues and appropriate questions for the questionnaire, they 
will not be discussed in detail here* Instead, they will be discussed 
when appropriate to provide clarification of the responses to the survey* 
A few general comments, however, may be useful* 

I 

Before beginning the interviews we had anticipated some demand 
for subject-specific writing exercises Ci*e*» exercises that required 
expertise in a particular subject-matter area)* We found no support for 
the inclusion of such exercises on an admissions test* There was near 
universal agreement among faculty members that subject-matter knowledge 
could be assessed with other testing formats, and that subject-matter- 



^^Advisory committee members were Louis Arena (University of Delaware) » 
Hunter Breland (ETS) , Roberta Camp (College Board, ETS), Diane Larsen- 
Freeman (School for International Training, Brattleboro, Vt*)» Charles 
Stansfield (TOEFL» ETS), Barbara Suomi (TOEFL, ETS), and Barry Taylor 
(University of Pennsylvania)* 
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specific writing skills (e.g., formatting lab reports ci preparing 
business case studies) would be part of the course of instruction rather 
than prerequisite skills. Nevertheless, certain kinds of writing were 
seen as more important in some subject-matter ar^ ^ than others. (For 
example, many business faculty were particularly concerned with the 
ability to write persuasively.) Therefore, the questionnaire did not 
attempt to deal with writing exercises for testing specific subject-matter 
knowledge; it did include sections on the kinds of writing that might be 
of varying Importance in different academic disciplines. 

Because the questionnaire focused on problems of students in their 
first year of study, problems in the writing of theses ^re not addressed. 
However, it is worth noting that a number of engineering faculty indicated 
that the thesis presents a major hurdle even though students can success^ 
fully complete their first year doing little or no writing. Faculty 
members from two institutions stated that many foreign students (and some 
native-speaking students) essentially have their dissertations written for 
them; although the students provide the conceptual framework , the actual 
writing is done by someone else. 

Questionnaire content . The questionnaire contained six major 
sect ions. Section I asked for some simple descriptive background 
information about the department. Section II surveyed wr it ing-^task 
demands, .Section III investigated criteria used to evaluate written 
assignments. Section IV obtained data on writing problems of native 
and nonnative students. Section V solicited information on the use or 
potential use of a writing sample in the admissions process, and Section 
VI asked for acceptability ratings on ten specific topic types. A copy 
of the questionnaire for graduate departments is provided in Appendix A. 
The same questionnaire was sent to undergraduate English departments, 
eajcept that the word ''undergraduate" was substituted for the word 
"graduate" throughout. 

Most items are self-explanatory and require no further elaboration. 
Note, however, that the "criteria" categories in Section III and the 
"problem" categories in Section XV are Identical. Some ol the general 
features listed were adapted from West ( 1982), who successfully used 
similar dimensions when he conducted a survey of engineering college 
faculty members at the University of Florida. These categories appear to 
provide an adequate description of the general features of writing while 
remaining free of linguistic jargon that might not be understood by 
faculty members who are not writir^ specialists. During the course of our 
interviews, in fact, we found that faculty members in disciplines other 
than English understood and were comfortable with the terminology and 
classifications used in the questionnaire. 

The questionnaire instructions asked respondents to make judgments in 
the context of experiences and expectations for the firs't year of educa~ 
tion in their institutions, gradu*ite or undergraduate. This instruction 
was based on the premise that the primary concern at the time of admission 
is v;lth th^ degree of preparation of the candidate, at entry level, that 
will enable him or her to succeed in a given educational program. Beyond 
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the first year, undergraduate and graduate coureework ie designed to 
help etudente develop skille that are more epecifically oriented toward 
their choeen fields; in the area of writing* the different programe taay 
emphasize skille in discourse modee and content areas that are more 
specialized. Expecially in^undergraduate programe, writing*taek demande 
vary considerably ae students select and change their major coureee of 
etudy. Respondents aleo were asked to attempt to reply to the question**- 
naire from the perspectives of their specific departments, rather than 
from their perspectives as individuals within depart^pents. 

Development of topic types . The topic types pre?»ented on pages 8*16 
of the questionnaire emerged from in-depth discussions with advisory 
committee members, research and program staff member^? at ETS, and faculty 
and administrators at local institutions. This approach of using topic 
types was selected in preference to asking respondents to rate various 
desired attributes of writing sample tasks. Appearing on a list, certain 
abstract features might be more appealing than others, but we assumed 
that, when confronted with concrete examples of actual tasks that might be 
required of candidates, respondents would be able to ap^^raise the task 
demands of a particular writing sample more realistically. In reacting to 
the topic types, respondents were instructed to respond to the writing 
communication demands and the skills likely to be elicited by a type of 
task, rather than to the specific wording or subject matter of an example. 
The topic types were structured to demonstrate how a stimulus for a 
writing sample might elicit particular writing skills. Thus it was 
ne ce ssary to focus the judgments of respondents on the value of the 
information that might be obtained from a type of sample rather than on 
details about the content or prose of the stimulus. (Eventually, as 
writing samples are designed for research or implementation purposes, 
these details about topic choices will need to be carefully considered.) 

Ten different topic types were selected to provide a concrete and 
thorough overview of the possible task demands that a writing sample might 
elicit. Two specific examples were provided for each topic type. As the 
examples were prepared, we attempted to eliminate as much cultural bias as 
possible by using content that might be construed as ''universal** rather 
than as oriented to Western culture, but with the recognition that certain 
conditions of our culture must be faced by international students who 
attend our institutions. The examples also were phrased so that they 
would specify clearly the task that was required of the student writing 
the sample.* 

The topic types range from the more basic modes of exposition to 
somewhat more complex writing task demands. They also vary in terms of 
the concrete nature of the rfiquirements; the early types clearly require 
straightforward description and no more, whereas the last two types are 
somewhat more open-ended, allowing the writer to demonstrate a variety 



^Roberta Camp, a member of the ETS professional staff who has considerable 
experience in designing writing sample stimuli, assisted us in preparing 
the topic type examples. 




of possible writing ekille to the extent that he or ehe can do eo. All 
the topic types attempt to elicit some portion or portions of the kinds of 
writing skills that might be expected of students at entry level, or in 
early training in their academic fields. The following paragraphs briefly 
describe the assumptions about writing task demands that underlay the 
selection of the topic types (see Appendix A, pages 8-^16, for actual 
topics) . 

Type A. Personal Essay 

In some respects, this type of stimulus presents a simple writing 
task, in that candidates might be more comfortable writing from their 
own experiences, and can produce *'good** samples of writing skills. 
The skills that can be elicited, however, are possibly too basic, 
and may not serve as good indicators of the abilities required by 
academic writing. The personal essay employs a subjective component, 
whereas objectivity tends to be emphasized in many of the academic 
disciplines. The comments obtained during interviews and on the 
questionnaires also point out that many international candidates may 
have memorized personal statements, thus defeating the purposes of an 
extemporaneous writing sample. Some of these personal essays may be 
redundant with the personal statements obtained in the admissions 
process, as well. Furthermore, the personal essay may present a 
disadvantage to nonnative speakers of English who are intimidated by 
the prospect of honest self-revelation to readers of their samples, 
fearing retribution from these in their own countries or in their 
chosen U.S. or Canadian institution. Finally, candidates from 
certain cultures may be unable to write personal ' essays because this 
form of exposition is either not sanctioned in their societies or is 
not a form with which they have experience. 

Type B. Sequential or Chronological Description 

As with the personal essay, this type of task requires a relatively 
simple form of written production. The writer needs only to present 
a picture with words, and is not required to rely upon other modes 
of discourse to meet the task requirements. This kind of description 
eliminates the personal element, in that the student need not reveal 
any personal insights or biases that potentially pose a threat from 
the evaluator of the composition.. The particular topic that is 
assigned may create a problem, however, because the international 
student may lack sufficient experience to draw upon. This foim of 
descriptive task may reflect important academic or f ield^related 
Skills, but may elicit skills that are so elementary that they 
require little organization or analysis because the topic itself 
supplies the natural organizing elements. 

Type C. Spatial or Functional Description 

This kind of topic presents the advantages and disadvantages of Typ9 
B tasks, yet may elicit more organizational akllls (such as relating 
th3 parts to the whole) than does the sequential or chronological 
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deecription. Again, the topics may pose "universal" situations, but 
they may also be trite, lacking the degree of depth that requires 
thoughtful analysis and coherence. However, this topic type has face 
validity, in that academic or professional writing frequently calls 
for this kind of writing. 

*^y_p 0 D* Compare and Contrast 

On a slightly more complex level of discourse, topics such as these 
require exposition as well as description. Since the student is not 
required to take a position, the possibility of perceived threat 
presented by revealing personal opinions is eliminated. As with 
topic types B and C, this type may reflect a kind of writing task 
that may be required of students, but it represents only one fragment 
of the several writing skills that characterize academic or profes-* 
sional writing. 

Type E. Compare and Contrast Plus Take a Position 

Although incorporating the positive elements of Type D topics, 
the added requirement of taking a position may penalize certain 
cultural groups. In many fields, however, argumentation blended with 
description and exposition is the predominant mode of discourse. In 
business, psychology, and civil engineering, for example, the student 
must go beyond a statement of facts to support the validity of one or 
more propositions. The writing of this kind of composition may 
elicit the student's skills in thinking about and organizing facts 
and ideas and may be considered a "better" representation of what he 
or she can do with a subject. With this type of sample, however, the 
student may not perform as effectively, especially when presented 
with a more general topic as opposed to a more familiar topic in 
the field. 

Type F. Extrapolation 

This type of topic calls for a blend of the several modes of 
discourse, as in Type E, in addition to an element of "creativity" 
or imagination. Because some degree of personal experience or 
judgment needs to be injected beyond basic description and exposi*' 
tion, this kind of task may introduce cultural bias. The demands 
of the task, though, represent a facet of the kinds of thinking 
and coherence in writing that parallel academic and job^^related 
assignments. One limitation of the extrapolation task is the amount 
of thought and organization required in the prewriting stage; time 
limits may restrict adequate development of a topic and eJ icit a 
sample that is not truly representative of a student's skills. 

Type G. Argumentation with Audience Designation 

This type of topic elicits skills similar to those elicited by the 
Compare and Contrast Plus Take a Position task (Type E), yet it 
introduces the valuable constraint of addressing the composition 




to the appropriate atidlence. Ideally^ every writing jample stimulus 
should designate an audience In order to communicate more prt^cisely 
the purpose of the composition, hence the skills to be demonstrated. 
In reality, however, many native and nonnatlve speakers of English 
have difficulty with preparing a piece that is directed to a specific 
audience. Instead, most undergraduate and beginning graduate 
students write for an audience of one, the Instructor. At the 
graduate level, training In certain fields does Involve the orienta- 
tion of students toward audiences with whom they are likely to 
Interact. However , graduate Instructors report that even 
professionals In their fields tend to deal Inadequately \iith audience 
designation. 

Type H. Describe and Interpret a Graph, Chart, etc . 

This task Is designed to elicit the skills of description and 
exposition. In addition, the student Is asked to "Interpret" the 
Information that Is presented plctorlally. The task of lntc:rpreta- 
tlon presents an open-ended Instruction, In that student responses 
might range from contributing slmpl Is tic , mechanically organized 
lists or Impressions to demon<!tratlng some of the skills of well- 
organized argumentation. Therefore, this Instruction may elicit 
writing samples that reflect thinking as ^11; hov^ever, the student 
who more strictly follows the Instruction may not exhibit the 
thinking and cohesion of Ideas that may be within his or her 
repertoire of skills. The demands of this task also may penalize 
students who lack graph-reading or spatial ability, or for whois an 
"Interpretation" may reflect a non-Western, cultural difference 
(although these biases could be largely prevented with carefully 
designed stimuli and/or scoring systems). Certainly, many fields 
require at least an elementary level of graph- or chart-reading 
ability that can be approximated with well-chosen topics. This 
type of topic reflects another aspect of the kinds of academic and 
professional skills that are important, but It does not necessarily 
present a complete measure of available skills. 

Type I. Summarize a Passage 

Many disciplines require the combination of abilities to read, 
comprehend, extract main Ideas, and summarize In one's own words. On 
the other hand. International students may be penalized particularly 
by the content and vocabulary and by an Inability to comprehend the 
full and Implied meaning In the passage. As a result, the writing 
samples obtained with this task might consist of lists or paraphrases 
that mimic the thinking and organlzatl\>n of the author o£ the 
pass^age rather than reflect those of the otudent. The determination 
of a passage for this kind of writing stimulus requires the careful 
selection of an optimal reading level, "universality" of the subject 
matiier and vocabulary, a well-written and clearly organized piece of 
prose, and precise Instructions that communicate the task demands. 
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Type J. Summarize a Passage and Analyze/Assess the Point of View 

Going beyond the skills elicited by Type I topics, this type of topic 
increases the demands on the writer by including an analysis/assess- 
ment component. Along with the drawbacks of Type I topics, this 
additional skill presents a more open-ended challenge — the student 
may demonstrate the skills at his or her command in toing beyond 
basic summarization. It allows the writer to demonstrate organiza* 
tion and thirling with the option to compare and contrast, to take a 
position, or to extrapolate .beyond the given information. Time 
limits impose an unreasonable restriction, however, since the task 
requires sufficient time to read and digest and to think and organize 
in the pr€»writing stage. Thus the task may convey expectations that 
are too complex to be achieved within a brief time i. terval. 



Sampling Plan 

To obtain a representative sample, we selected institutions that ^et 
the following criteria: 

1. Institutions with enrollments of foreign students* exceeding 
1,000 

2. In the United States, four geographic areas with the highest 
proportions of foreign students enrolled (assuming that this 
geographic distribution would provide a reasonable distribu- 
tion of students from different foreign countries) 

3. A distribution of public/private institutions approximately 
proportional to the public/private distribution of schools in 
the United States that foreign students attend (i*e., achieving 
a balance between private schools typical of the Northeast with 
public schools typical of the Midwest) 

4. In Canada, institutions with high proportions of foreign students 
enrolled 

The i^nstitutions were selected on the basis of these criteria from 
lists compiled in Open Doors: 1980/81 (Boyan, l98l) and further supported 
by TOEFL program statistics. (See Arjpendlx B for enrollment data for the 
institutions that participated.) All of the Institutions that were chosen 
had enrollments of foreign students exceeding 1,000, except for one school 
In the Pacific region with an enrollment of 982. The regions in which 
most foreign students were enrolled were the Northeast, Midwest, Pacific, 



*Data from Open Doors: 1980/81 were used as a basis for selecting the 
sample; since that text uses the term "foreign students,** the term is 
used in this section de^'^ribing the sampling plan. Hereafter, our 
text uses the term **nonnative speakers of English** to more accurately 
identify the focus of our study. 
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and South/Southwest In the United States. Although Open poors does not 
Include Canada, we also decided to Include three Canadian Institutions 
that enrolled large numbers of foreign students. Of the seventy United 
States Institutions reporting enrollments of over one thousand foreign 
s,tudents In 1980/81, we decided to ootaln a sample of at least thirty. In 
addition to the Canadian institutions. Lists of eight to ten institutions 
in each of the four regions of the United States were prepared, with the 
expectation that not all of the institutions would be willing or able to 
participate. 

^^cause it would not be practical to sample from every academic 
discipline that enrolls foreign students, we selected seven disciplines 
("departments") as the fields of study in which most foreign students tend 
to enroll and tbat also represent contrasting areas of education. Since 
p. TOEFL contributes to decision making aL the time of admission, we decided 
to focus on the skills of students at entry level and during their first 
year of education as undergraduate or graduate students. The interviews 
suggested that most writing by first^year undergraduate students was done 
in English courses; thus the undergraduate English departments were 
selected to provide data representing initial undergraiiuate experiences. 
Graduate departments with heavy concentrations of ioreign students were 
selected from data provided in Open Doors and from recent Graduate Record 
Examinations background questionnaire responses (Wilson, 1982). At the 
graduate level, the following six departments were chosen: electrical 
engineering, civil engineering, computer science, chemistry, master of 
business admlni6tra,'^iu'\ (MB/J programs, and psychology. Although graduate 
departments in the social sciences typically report relatively low enroll- 
ments of foreign students, this area was Included because it was felt that 
resultt> for the social sciences might be markedly different from the 
results in business, "hard" science, and engineering. Psychology, the 
social science department with the highest nonnative enrollment, was 
selected. 



Data Collection Strategies 

To ensure a high rate of return, we telephoned faculty members or 
administrators at all institutions on our list to obtain the assistance of 
one individual at each Institution who would distribute and collect the 
questionnaires for each of the seven disciplines. These potential 
coordinators" were identified through the NAFSA Directory (1980). Direct 
telephone contacts with these individuals, or with Individuals who had 
replaced them in their positions, enabled us to make arrangements with 
coordinators who were willing to participate* Using this approach, we 
also v .rc assured that the coordinators would select faculty members, one 
in each of the seven departments, who were most familiar with the academic 
demands of their departments (institutions, in the case of undergraduate 
English) on native- >and nonnative speakers of English. In addition, these 
contacts enabled us to determine whether all seven departments existed at 
each school, so that we might mall the appropriate number of question- 
naires; if the coordinators were in doubt about specific departments, more 
questionnaires were mailed than might be needed* v 



27 



-19- 



As soou as a coordinator at each institution vu/; cotif ir^ted (listed in 
Appendix C) , a package of materials was mailed. Each packagi:: contained 
the fol lowing : a letter containing informat iot: ebo^jtt the study and 
procedures for the coordinator, a sample of the und<:,rgraduate and graduate 
questionnaires for the coordinator, letters explaintfx^ the stu^y for each 
faculty member, questionnaires and honorarium forms for t^ach faculty 
member, and a postage prepaid mailing envelope foe returning the 
materials. (The letters appear in Appendix D. ) For the institutions that 
agreed to participate early in our telephone conversations, one month's 
time was allowed for the completion and return of the questionnaires. 



ERIC 




Section 111, Results 



Results are presented In the same order In which the questions appear 
In the questionnaire. 'Hie reader should refer to the exact phrasing and 
format for each question on the questionnaire before reading the results 
for that Item* 



Sample Description 

Usable questionnaires were r^^calveu from a total of 190 academic 
departments from 34 universities, which represents 82% of the 231 <]ucs- 
tlonnalres sent to 36 schools. A total of 213 ^Questionnaires (92% return) 
was actually returned by local coordinators , but some were returned 
blank because the faculty from departments with very low foreign student 
enrollments felt that they had Insufficient Information to complete the 
questionnaire. Four questionnaires from departments at Canadian univer- 
sities were dropped because the "nonnatlves** In those departments were 
bilingual French Canadians who did not fit the typical nonnatlve pattern. 
Only two Institutions (both private Eastern universities) failed to 
return any questionnaires. Some universities did not have academic 
departments In all of the seven areas, and a few departments did not 
r^iturn questionnaires «ven though jother departments at their universities 
did. Thus, the Ns \ary slightly from department to department. 

Table 1 provides Information on the number of questionnaires received 
from each academic discipline and summarizes the background Information 
from page 1 of the questionnaire.. The percentage of nonnatlve students In 
a department ranges from a low of 6% In psychology to a high of about 50% 
In both englr.eerlng fields. Note that graduate management departments are 
typically much la^'ger than other graduate departments; thus, the mean 
nunber of nonnatlve students In management programs Is comparable to the 
number In engineering programs even though the percentage figure Is much 
higher In engineering- The plurality of the departments In each academic 
discipline reported that most nonnatlve students came from South and 
East Asia* 



Writing Task Demands 

Writing task demands were assessed In two ways. First, faculty 
In each department ;^re asked to Indicate the number of times per semester 
that each of a variety of tasks would be assigned to flrct-year students 
In all of their courses (see page 2 of the questionnaire In Appendix A)* 
Second, faculty were asked to rate, on a scale of one to five, a variety 
of writing activities In terms of, liaportance (see page 3 of the question- 
naire)* Results of the section concerned with frequency of assignment 
(questionnaire page 2) are summarized In Table 2. In this table, and In 
subsequent tables, the percentage of departments rather than number of 
departments was used to facilitate comparisons across academic disciplines 
that were represented by differing numbers of departments. Percentages 
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sometimes fail to add to 100% because of omitted responses. Omits 
appeared to be random, with never more than three omits on any item for 
any of the academic disciplines. 

Frequency of assignments . Lab reports or descriptions of experiments 
appear to be assigned to first-year students at least once per semester 
in the graduate departments sampled » except for graduate manageuent. 
However, lab reports for first-year undergraduAte students are apparently 
relatively rare. Writing brief summaries of articles appears to cut 
across academic departments, but it is far from a universal demand. At 
least 25% of the departments in all fields except psychology and computer 
science indicated that their students are never given brief article 
summary assignments* Brief research papers (five pages or less) appear 
to be relatively common across fields, but longer research papers are 
required in fewer than half of the electrical engineering and chemistry 
departments. 

Despite the emphasis on creative writing (fiction, poetry, drama) in 
many elementary and secondary schools, this kind of writing is apparently 
relatively rare at the university level. Sixty-two percent of the English 
departments reported that undergraduates receive no creative writing 
assignments in their first year. At the opposite extreme, for under- 
graduates, expository or critical writing assignments are reportedly quite 
frequent. Ninety-seven percent of the undergraduate programs (31 of the 
32 in the sample) reported at least three expository or critical writing 
assignments per semester. However, this kind of writing is relatively 
rare in most of the graduate departments. Business management was the 
only graduate field in which fewer than half of the departments reported 
no assignments in expository writing. Exams with essay questions appear 
to be fairly common on both the undergraduate and graduate levels. An 
exception is electrical engineering, where over half of the departments 
reported using no essay exams. This is consistent with findings from the 
interviews. 

The electrical engineering faculty members interviewed reported that 
most of their exams are problem sets that could be answered with strings 
of mathematical symbols. The interviews with civil engineering faculty 
indicated that they use some purely mathematical problems on exams, but 
that two or three questions requiring essay responses are usually included 
as well (e.g., "Describe the advantages and disadvantages of highway 
route A versus route B**). The questionnaire responses reinforce this 
distinction between civil and electrical engineering. 

Group writing exercises are common in graduate management programs, 
but are relatively rare elsewhere. The interviews indicated that in some 
HBA programs practically all writing assignments are done on a group 
basis. Students with relatively poor writing skills but good s'lills in 
other areas may be able to get by in such programs if they are /ortunate 
enough to be assigned to groups that Include at least one good writer. 

Case studies are very common academic tasks in MBA programs, with 75% 
of the departments reporting that at least three case studies are assigned 
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per semester. Case studiee are reportedly assigned at least once per 
semester in 59% of the psychology programs. The questionnaire did not 
differentiate between clinical and experimental psychology progr^s, but 
there would undoubtedly be important differences between the two. 

Importance of writing skill to succes s . During the interviews , 
several engineering faculty members said that they put very little 
emphasis on writing in the graduate program and that writing skills 
are not important for su^ccess in graduate srhool. However* these same 
professors indicated that the most successful practicing engineers are 
th^se who write well. To determine the generality of these beliefs about 
tne Importance of writing skill to .success in school and after school, two 
items were included on the questionnaire (items 1 and 2 on page 3). The 
results are summarized in Table 3. Except for psychology and under- 
graduate English, writing was consistently rated as more important to 
success after graduation than to success in school. As suggested by the 
interviews, the difference was particularly striking in engineering. In 
both civil and electrical engineering, only about 20% of the departments 
rated the importance of writing to success in school in one of the two 
highest categories (4 and 5), but over 60% of the departments in both 
areas rated writing as a 4 or 5 in importance after graduation. 

Importance ratings for writing skills . Seven writing skills were 
rated (1-5) for degree of importance (see questionnaire, page 3). These 
ratings are presented in Table 4. Results generally confirmed a priori 
expectation s. For example , describing an apparatus is relatively 
^ unimportant in MBA departments and relatively important for engineering 
and computer science. Describing a procedure is apparently especially 
important for computer science majors. Arguing for a particular position 
is very important for undergraduates and graduate business majors but 
is relatively unimportant for students in engineering, chemistry, and 
conqjuter science. Organizing arguments from several sources appears to be 
critical for undergraduates, graduate business students, and psychology 
majors. Analyzing or criticizing ideas, excerpts, or passages was rated 
as a particularly important skill for undergraduates, graduate business 
students, and psychology majors. 



Criteria Used to Evaluate Written Assignments 

Importance ratings of various evaluation criteria for written 
assignments (from page 4 of the questionnaire) are summarized in Table 5. 
These ratings reflect **the extent to which grades for written assigments 
in your courses for beginning graduate (undergraduate) students are 
influenced by each feature." In general, grammatical and sentence-level 
features (e.g., punctuation, spelling, sentence structure) were rated as 
less important than more global^ essay characteristics (e.g., par^gr^ph 
organization, paper organization, qualit^y of content, development of 
ideas). However* the undergraduate English departments rated sentence 
structure as an important grading criterion (94% rated it 4 or 5). 
Appropriateness to audience is another factor that is important to 
undergraduate English departments (85X rated 4's or 5*s) and graduate 
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TABLE 4 

- IMFORTAHCE OF VARIOUS tmiTIHG SKILLS 

DISCIPLINES* 



VrltlDg Skill laportaoce U6E HIIA CE EE P C OS Total 
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management programs (65% rated 4*e or 5*s), but much . les^e Impor tant 
In engineering (20% and 28% rated 4's or 5's In civil and electrical 
engineering, respectively). A graph of the mean rating on each feature 
for each department clearly llluetratee the trende dlecuseed above* In 
Figure 1, the means for most academic disciplines fall within the rectan~ 
gular box above each of the rating categories* Means for disciplines 
falling outside this range are plotted separately* The figure clearly 
shows the generally higher ratings for the global characteristics as well 
as the extent to which ratings for undergraduate English departments 
and MBA programs were atypical* ^ 

Question 15 on page 4 of the questionnaire asked which of the 
features was most Important and which was least Important* The results 
are summarized In Tables 6 and 7* Consistent with the results presented 
In Table 5 and Figure 1, most departments rated one of the discourse- level 
characteristics as most Important and one of the word or sentence-level 
characteristics as least Important* Particularly striking Is the number 
of departments that thought that size of vocabulary Is least Important; 
more departments selected this choice than any other option* This result 
should be of Interest to students (both native and nonnative) who use 
thesauruses to sprinkle obscure words through their writing* 

Table 8 Indicates that most departments reportedly use the same 
standards to evaluate the writing of native and nonnative speakers of 
English (questionnaire page 4, Item 15)* However, In each field except 
psychology there was still a significant mlno^^lty of departments using 
different standards* Departments answering "No" to item 16* (l*e*, 
de par tments that use different standards to evaluate the writing of 
nonnative students) were asked to indicate which ' :ltlng features they 
evaluate more leniently for nonnatlves* These re^ -ilts are presented In 
Table 9 (page 35)* Most of the features evaluated more leniently were at 
the level of paragraph organization or at a more molecular level (e*g«» 
punctuation/ spelling, sentence structure, vocabulary size, vocabulary 
usage), but overall writing ability Is evaluated leniently by many depart- 
ments, as well* Even among those departments that use different standards 
for nonnatlves , the areas of paper organization, quality of content, 
development of Ideas, adequately addressing the topic, and meeting assign- 
ment requirements are generally evaluated with the same standards for 
natives and nonnatlves* Note, however, that appropriateness to audience 
Is one dlscourse**level characteristic that Is frequently evaluated more 
leniently for nonnatlves* 
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*"Do you use the same standards to evaluate the writing of native and 
nonnative speakers of English?" 
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FIGUKE 1 

GROUP MEANS ON EVALVATtON CRITERIA 
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Evaluation Criteria (Nuid)ers correspond to question numbers on page 4 of the questionnaire.) 
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Kot«: Except as noted* neaos for all departments are within boxed area. 
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TABLE 8 

Percent of Departments Indicating Use of Same 
Standards for Native and Nonnative Students 

Disciplines 



UGE MBA CE EE P C CS Total 

75 55 65 84 42 86 76 69 



Writing Problems of Native and Monnative English-Speaking Students 

The features of written assignments on page 4 of the questionnaire 
were repeated, and respondents were asked to rate perceived problems of 
both native and nonnative speakers of English* on a three^oint scale (1 
for minor or rare, 2 for moderate or occasional, and 3 for large or 
frequent). In general, judgments about these problems were in close 
agreement across departments (see Table 10)* 

The most prevalent trend indicated across the several features, for 
all departments, was that nonnatives are perceived to have large problems 
with overall writing ability, as well as with correctness of punctuation/ 
spelling (48-85% range of percentages) and with quality of sentence 
structure (42-84% range)* These three features generally were rated as 
moderate problems for natives* Three other features — quality of content, 
addresses topic, and meets assignment requirements — were not reported to 
be large problems for either natives or nonnatives* 

Although some slight divergence of opinion was expressed by specific 
departments for each of the twelve features, the ratings assigned by most 
departments can be summarized as follows: 

o Large problems for nonnatives/moderate for natives — correctness 
of punctuat ion/ spelling, quality of sentence structure, and 
overall writing ability 

o A large or moderate problem for nonnatives/moderate for natives — 
quality of overall paper organization 

o A large or moderate problem for nonnatives/moderate or minor for 
natives — size of vocabulary 

o A large or moderate problem for nonnatives/minor for natives — 
appropriateness of vocabulary usage 



*For brevity, the terms native and nonnative speakers of English are 
truncated to **nat ive** and **nonnative , " wi th ''speakers of English'' 
implied* 
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TABLF 9 

VKinUG tEATinURS MWK LEKIOITLT BT BEPAItlNENTS 

TBAT USE iriFmSfT STAMUUtitf fOI MIQUnVC STtnVHTS 
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CS 
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90 
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58 
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79 
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d3 


3. 
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60 
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5. 
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13 


17 


67 


25 


38 


75 
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3S 
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0 


22 


25 


14 


50 


20 
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7. 
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0 
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0 


0 
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1. 
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0 
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0 
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9. 
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13 


50 


SB 


50 


50 


50 


80 


55 
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0 


25 


0 


0 


0 


3 
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50 


25 




25 


38 


50 


60 


41 


u. 
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0 


0 


0 


0 


7 


25 


0 


3 



^ Boo Tbblt 1 for V» and lobol 4»»crlptloo», 

^ All ttblo uitrlfto oro ptrccototM of tb» totol ntnibtr of dttpmneott In oocb ocodofeic dUclpllnt tbot ontvorod 
Ho** to Itoa 16. r«9ordlAs u»o of tlalUr crltorU for oofmotlvo and tkotlvo opoolcort of Aitli^b. 
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TABLE 10 

URinNG PBOBLEIfS OP HATIVE AHD KOKKATIVE STUDENTS 



DISCIPLINES^ 



?c*tur« ^It^^r* OCE HBA CE EE P C CS TOTAL 

miQg NNN NNM NNM MNK NNN UNM MtQl MNM 
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*i o A moderate problem for nonnat Ives /moderate or minor for natives**" 
meete assignment requirements 

o Moderate problems for both nonnatives and natives — quality of 
paragraph organization, quality of content, development of ideas, 
and addresses topic adequatelj^ and directly 

o A moderate problem for nonnatives/minor or moderate for natives" 
appropriate to the audience 

The undergraduate English departments tended to express the most 
disagreement with other departments, possibly because they are more 
sensitive to particular features of writing. For example, English depart- 
ments perceived nonnatives to have large problass with quality of sentence 
structure, appropriateness of vocabulary us^^e, and adopting a tone, 
attitude, or style appropriate to the audience. For both natives and 
nonnatives, many English departments rated quality of paper organization 
and development of ideas as moderate to large problems. 

To enable us to provide an overview of the perceived native-nonnative 
differences with regard to writing problems, the differences between all 
assigned ratings were calculated. The differences in Table 11 were ^ 
obtained by subtracting the ratings for nonnative speakers of English 
from the ratings assigned to native speakers. The distribution of these 
differences (native ratings minus nonnative ratings) ranged from -2 to +2 
for each feature for departments within each discipline. The differences 
were summarized by assigning a 0 where a large percentage of departments 
indicated essentially no difference, and assigning an X where a large 
percentage indicated a difference (+2 or +1). In cases where the percent- 
ages are essentially split between assignments of no difference-, fference 
ratings, both an X and an 0 were tabulated. This tabulation provides 
a more succinct overview of the judgments regarding native/nonnative 
differences. For example, 72% of the MBA departments perceived a 
difference between natives and nonnatives in the quality of sentence 

structure, resulting in an X tabulation; for undergraduate English 
^departments, however, 34% perceived no differences (0)* and 60% perceived 
differences ( + 1) for this feature, resulting in an X,0 tabulation. This 
tabulation compresses the table of percentages (Table 10), providing a 
more succinct overview of the judgments. 

No differences (0*s) in the problems of natives and nonnatives were 
reported, for all departments, for the following features; 

6. Quality of paper organization 

7. Quality of content 

8. Development of ideas. 

10. Addresses topic adequately and directly 
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^ Set Ttble 1 for Ub«l detcrlptlong. 



Obtained hy eubtrectlttg the retinga eealgned to nonnetlve apeakere of EnKllah froff the ratlnga easlgncd to native epeakers 
pf Eogllah. In Qc Ittstaaoe were native ptoblema report«l to be larger than the proble«a of nonnatlvea. Since entrlea for 
the '*Other" category ^re vlnlsal, ranging froM 89.66*100Xr theae entrlea are not reported* 

gey t 0 * no difference 

X - difference (nonnatlvee vlth larger problem than natlvee) 
OfX - apllt between no difference end difference 
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Essentlally no differences were reported, with a few discrepant 
departments, for the following problems: 

5. Paragraph organization (with only civil engineering depart* 

ments reporting nonnatlve problems) 
11. Appropriate to audience (with ties for differences/no 
differences In undergraduate English and MBA departments 
[O.X]) 

Finally, differences (X's) in problems between natives and nonnatives 
were reported for the following; 

1. Correctness of punctuation/spelling (with some ties) 

2. Quality of sentence structure (with one tie in psychology 
departments) 

3. Size of vocabulary 

4. Vocabulary usage 

9. Overall writing ability (with 3ome ties) 

In no instan'^e were the problems of natives reported to be larger 
thrn the problems of nonnatives. Since entries of natives for "other" 
categories were minimal, in that blanks (no entries) ranged from 89.7 to 
100%, these few entries are not reported. 

One interesting observation is that, in most departments, overall 
writing ability was judged to be more of a problem for nonrtitives than 
for native speakers of English, yet the dominant problems reported for 
nonnatives are at the sentence level (feature 2), or deal with vocabulary 
(features 3 and A), and punctuation/spelling (feature 1). In contrast, 
for the ratings reported for Criteria Used to Evaluate Written Assignments 
(page 4 of the questionnaire) , all departments except undergraduate 
English claimed to place more emphasis on features beyond the sentence 
level — quality of overall paper organization (feature 6), quality of 
content (feature 7), development of ideas (feature 8), addresses topic 
(feature 10), and meets assignment requirements (feature 12). Hypothet- 
ically, the problems related to these features are detected but not used 
as ultimate criteria; however, the fact that these problems are emphasized 
in the context of the overall writing ability of nonnative speakers should 
be noted. 



A Writing Saiiiple in the Admissions Process 

1 

' Pa^e 6 of the questionnaire contains items concerned with informa- 
tion that is used, and information that wuld be desirable, at the time 
of admissions , particularly writing samples for nonnatlve speaking 
applicants. 

Writing samples used at the time of admission . Item 1 asked whether 
the respondent's institution administered a writing sample, and for 
what purposes. For the total sample of departments comple ting the 
questionnaire, only 27 percent indicated that they use a writing sample at 
the time of admissions (Table 12). Nearly half (A7%) of the English 
departments use a writing sample. The use of writing samples by the 
other departments ranged from 23 to 28 percent. 
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TABLE 12 

PURPOSES OF WHTIHC SAMPtES CUEREKTLT USED AT THE TIME OF ADHISSIOHS 
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^ Ail table «ntrle« ara percentagei of the total nunber of d<p«rt»enta In each academic dliclpllna. 

ICej: tl * native apeakera of English 
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For thoee departments that administer a writing sample, a prevalent 
purpose v^povt^d across all departments was for making borderline admis- 
sion^ decisions about both natives and nonnatives, particularly in the 
chei^istry and computer science departments. Some of the departments 
that administer a writing sample also use it as an essential admissions 
criterion with the exception of civil engineering. Instead, A percent of 
the civil engineering departments use a writing sample for borderline 
admissions decisions for both natives and nonnatives; for nonnatives 
in particular, the writing samples are uded by these departments for 
course placement decisions (19% of the 23^!C reporting). Some electrical 
engineering departments (16% of the 20% repor*:ing) use the writing samples 
as an essential admissions criterion for nonnatives, whereas only A% of 
these departments do so for native speakers of English. The undergraduate 
English departments place the greatest emphasis on the use of writing ^ 
samples for course placement for both natives (31%) and nonnatives (41%). 
In addition to some civil engineering departments (19%), a few computer 
scienc^ departments use the writing samples for course! placement of 
nonna^Uves (12%). Very few departments use writing samples for job 
placement; 8% of the electrical engineering departments reported this 
purpose for nonnatives (4% for natives), and only 3% of the chemistry 
departments use wri ing sample for nonnatives. The "other" purposes/ 
reported by respondents were miscellaneous, minor entries. 

Other admissions information . ■ In response to item 3 on page 6 of the 
questionnaire (an open-ended question), 59% of the respondents indica^-ed 
what other forms of admissions information, about nonnative-speaking 
applicants would be helpful beyond the admissions criteria currently in 
use (Table 13). These responses were coded, "yielding fourteen other kinds 
of admissions information that might be needed. Of the 112 departments 
responding to this item, 44'"83% indicated the need for additional infomar 
tion, particularly the MBA departments (83%). The two predominant kinds 
of useful information that departments reported they might need were 
information about oral communications skills, language abilities, and a 
"general" writing sample (as opposed to a writing sample with a sp^^cific 
designation, such as modes of discourse,' in undergraduate English). The 
chemistry (41%) and computer science (32%) departments reported the 
greatest interest in information about oral communications skills; some 
interest in these skills was expressed by the MBA (14%), electrical 
engineering (12%) , and psychology (12%) departments. The '^language 
abilities" category was applied to code any responses concerned with 
more than one mode of communication. Approximately the same degree of 
interest that was expressed in oral communications skills was expressed 
for language abilities, even more so«by electrical engineering departments 
(20%) > but with less concern for abilities other than oral communication 
in the chemistry and computer science departments. 

As seen in Table 13, a variety of other needs at the time of adBiis* 
sion was reported by small percentages of departments in the several 
disciplines. Only 1 percent of the departments in the Scuiiple noted their 
concerns about the authenticity of writing samples for nonnatives (this 
problem also was occasionally mentioned ^y other respondents in the 
margins of the questionnaire). Primarily the undergraduate English 
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OTHER ADMISSIONS INPORMATICWl NEEDED ABOUT NONNATIVE SPEAXING E«CLtSH STUDENTS 
^"^ ' DISCIPLINES 
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defartments (25%) indicated an interest in a writing sample tor non- 
ndtivee; a little interest in a writing eample wae voluntarily exprese&d 
in this question by civil engineering (19%), MBA (14%), and psychology 
(12%) departments* A subsequent item, discussed in the next section, more 
directly addresses the degree of interest expressed in a writing sample on 
TOEIX. 

A TOEFL writing s ampl e * Item 5 on page 6 of tho questionnaire 
specifically asked the respondents to indicate whether a writing sample 
administered with TOEFL would be useful to them; 63% (N«120) of all 
departments checked **Yes**; 31%, or 59 departments checked "No" (Table 
14) « In particular, 97% of the undergraduate English departments and 83% 
of the MBA departments responded positively* Relatively large percentages 
(48*54%) of the other departments also reported an interest in a TOEFL 
writing sample* Of the purposes listed, 58% of the departments expressed 
some interest in using the wr itiag san.ple for borderline admissions 
decisions about nonnatives, especially the electrical engineering (7Z%), 
chemistry (66^), MBA (62%), and psychology (62%) departments* The under- 
graduate English departments (81%) particularly reported an interest in 
the use of the writing sample for course placement* Moderate to small 
percentaf^es of departm^ts also considered the purpose of the writing 
sam;>le h.o be a possible essential admissions criterion* Additional 
entries in the "other" blank covered a wide range of other possible 
purposes; the civil engineering entries, for example, suggested various 
ways to use a writing sample for "diagnostic** purposes* 

The sixth question on page 6 asked respondents to indicate which of 
two methods of scoring a possible TOEFL writing sample might be preferred* 
Of all departments responding, 58%' (N=l lO) reported an interest in 
separate scores for three global features of a writing sample; 31% (K»59) 
preferred a single score; 3% (n^5) checked both choices* Electrical 
engineering departments split their choices between single and separate 
scores, whereas considerable percentages of the other departments selected 
the option of separate sco res * These percentages of responses are 
reported in Table 15* 



TABLE 15 



Preferred Methods of Scoring 



Score 



Single 



Separate 
Score 



Both 



All departments 



UGE 
MBA 
CE 
EE 
P 
C 

cs 



34 
17 
31 
48 
25 
38 
24 
31 



59 
69 
54 
52 
62 
48 
60 
58 



6 

10 
0 
0 
0 
0 
0 
3 
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Additional comments * Respondents offered comments In the , final 
section of page 6, or in the margins throughout the questionnaire. These 
comments were recorded and sorted into common categories* Many of these 
notes addressed che coplc types that were evaluated on pages 8-16 of the 
questionnaire, or consisted of suggestions concerning the selection of a 
topic for a writing sample* These comments have been used to assist 
In the Interpretation of the ratings of topic types > and are either 
Integrated, or specifically referred to, In the results section dealing 
with these pages of the Instrument* The comments that are particularly 
pertinent to the Items on page 6 generally explained or reinforced the 
ratings che respondents supplied* They covered a wide range, Including 
holistic scoring, test security, and writing sample authenticity; various 
uses of a writing sample, and wanting to see the samples themselves; and 
the need to demonstrate the reliability and face and predictive validity 
of writing samples that might be administered on TOEFL* 



Evaluation of Topic Types 

Respondents rated each of ten writing sample topic types as 
ac ceptable/goodf acceptable/ f air , or unacceptable for use In making 
admissions or placement decisions for n on native speaking applicants* 
Results are summarized in Table 16* Topic types for which there were 
substantial differences across departments should be noted* F^r example. 
Type C (spatial or functional description) was rated relatively high by 
most departments but unacceptable by 34% of the undergraduate English 
departments* Comments from the English departments that disliked this 
topic suggest that they thought it discourages the eliciting of organi- 
zational skills; the task Is structured to such an extent that It could 
not easily be used to discriminate among students who can organize 
competently and tho^e who cannot* HowevF.r , this kind of structured 
description Is apparently very Important In electrical engineering and 
chemistry (where che arrangement of lab equipment must be specified In 
detail)* Type G (argumentation with audience designation) was unpopular 
with undergraduate English faculty but was very well liked by faculty In 
MBA programs* Curiously, It was apparently liked and disliked for the 
s^me reason, namely, that this Is a relatively difficult kind of writing 
to do* Many undergraduate English faculty members reported that It Is too 
much to expect beginning undergraduates, especially nonnatlves, to be able 
to do this kind of writing* However, the Interviews with business faculty 
Indicated that although they recognized that this Is a difficult writing 
assignment. It is also crucial to the kind of writing students have to do 
In MBA programs* 

Although Type H (describe and Interpret a graph/chart) was a highly 
rated topic In all graduate departments. It was rated as unacceptable 
by 56% of the undergraduate English departments* The English faculty 
complained that this topic type, like Type C, does not require the 
student to use organizational skills* In addition, they objected that 
It confounds writing ability with graph-reading skills* The graduate 
departments may have been less concerned about this confounding because 
they assume that students applying for graduate admission will have 



5Q 



Jkc«pt«bllity IbtlA^ for T«n Topic Tjp«» 







Ac copt ability 
Ratlnga 


tJGE 


HBA 


D 

CE 


I S C I 
EE 


F t I If E S 
F 


a 

C 


CS 


TOTAL 


Topic At 


£a M J — F « t aoQAl 


Ac c«p tab 1 e /Good 




34 


15 


24 


46 


31 


44 


34 






Acceptable/Fait 


38 


38 


65 


48 


50 


38 


36 


44 






tJnaccep tabla 


22 


24 


15 


20 


4 


24 


20 


19 


Topic Bt 


Desc t Ipt Ion— Sequ«ntl« 1 


Ac ce p t ab 1 e/Good 


31 


41 


42 


48 


29 


38 


48 


39 




or ChrODo logical 


Acceptable/ Fair 


44 


45 


46 


32 


54 


55 


44 


46 






tJnacceptable 


22 


10 


12 


12 


17 


0 


4 


11 


Topic Ct 


Duct Ipt Ion — Sp«tiftl 


Acceptable/Good 


25 


14 


31 


60 


42 


48 


36 


36 




or functional 


Acceptable/ Fair 


41 


55 


46 


24 


46 


38 


56 


44 






tJnacceptable 


34 


28 


23 


8 


12 


7 


4 


17 


Topic 


Co«q?« to/Cofit ru t 


Accept able/Cood 


75 


48 


35 


40 


33 


41 


44 


46 






Acceptable/Fait 


22 


41 


58 


40 


62 


38 


40 


42 






tJnacceptable 


3 


7 


8 


12 


4 


14 


12 


8 


Topic E: 


Coap«r«/Cof)tt««t 


Accep table/Gbod 


75 


86 


27 


28 


54 


31 


28 


48 




Plus T«k« « Foiitlon 


Ac cep tab 1 e /Fa Ir 


12 


7 


65 


44 


42 


45 


52 


37 






Unacceptable 


12 


3 


8 


20 


4 


17 


16 


12 


Topic ft 


Sxtt«pol«tloii 


AccoJ-ab le/Gbod 


47 


41 


19 


32 


25 


24 


?0 


31 






Acceptable/Fair 


34 


48 


65 


48 


62 


52 


64 


53 






tinacc ept ab 1 e 


19 


7 


15 


12 


12 


17 


12 


14 


Topic Gt 


Ar3uaent«tloQ 


Acc ep tab 1 e /Good 


41 


79 


46 


32 


29 


28 


12 


39 




trlth Audience 


Acceptable/Felt 


22 


7 


42 


32 


50 


38 


72 


36 




Designation 


tinacceptable 


38 


10 


12 


28 


21 


28 


12 


22 


Topic H: 


Deactlbe «nd 


Acce p tab 1 e/Gbod 


16 


59 


58 


56 


75 


66 


48 


53 




Intorptet a 


Accep toble/Falr 


16 


17 


31 


20 


8 


14 


24 


18 




Graph/Chatt 


tJnacceptable 


56 


14 


8 


12 


12 


7 


12 


18 


Topic I: 


Sumutlta a 


Acceptable/Good 


12 


31 


38 


20 


46 


31 


40 


31 




Faaaage 


Acceptable/Fal^ 


19 


38 


23 


20 


25 


28 


40 


27 






tinacceptable""^^ 


53 


14 


19 


40 


17 


21 


8 


25 


Topic Jt 


Sun«atlt« Flua 


Acc e p tab 1 e /Good 


31 


45 


42 


20 


58 


14 


28 


34 




Analyte 


Acceptable/Fait 


12 


24 


35 


12 


29 


38 


48 


28 






tJnacceptable 


41 


7 


15 


48 


4 


24 


4 


21 




^ See Tabla 1 fot Ha and label daacriptloao « 






iA eacn 











ERIC 



-47- 



acqulred the requisite graph-reading skills. Types I (summarize a 
passage) and J (sutDinarize and analyze a passage) were also seen as too 
complex by many of the undergraduate English departments and by faculty in 
electrical engineering* In addition^ several undergraduate English 
departments complained that these topics confound reading ability and 
writing skill. 

Most appropriate topic types * Respondents were asked to review the 
entire list of topic types and list^ in order> up to five preferred 
types « Because many departments listed only three pref e rred types » 
analyses presented here are limited only to firsts second^ and third 
choices. First^place choices are presented in Table 17. As anticipated, 
the results shown in Table 17 reflect the same pattern indicated in 
Table 16. Except for undergraduate English and MBA programs, the most 
favored topic type was the description and interpretation of a graph or 
chart (Type U). Business departments tended to favor argumentation with 
audience designation (Type G) and summary and analysis of a written 
passage (Type J). Undergraduate English departments were widely split, 
and there was no clear consensus favoring any single topic type. 

A scoring procedure was devised in order to consider simultaneously 
firsts second, and third choices. First choices were assigned a value of 
3, second choices a value of 2, and third choices 1. For each topic 
type, these values were multiplied by the percentage of departments 
selecting that type as a firsts second^ or third choice^ and the results 
were summed. For example, 19% of the undergraduate English departments 
selected Type A as a first choice (19 x 3 57); 6X selected A as a second 
choice (6x2^ 12); and 6% selected A as a third choice (6 x 1 6)^ for 
a total score of 75 (57 + 12 + 6 ■ 75). Scores could theoretically range 
from 0 (topic not selected as a first, i^ecord^ or third choice by any 
department in the academic discipline) to 300 (topic selected as the first 
choice by every department in the discipline). In fact» scores ranged 
from 0 to 167. Results are presented in Table 18. 

Although these results generally confirm the f j.ndings limited to 
first choices only (Table 17), some differences emerge from considering 
the first three choices. In particular, the preference of undergraduate 
English departments for Type E (compare/ contrast plus take a position) 
becomes clearer. The lack of enthusiasm for this topic type by civil 
engineering and chemistry departments remains apparent. Looking across 
departments > it is clear that no single topic type was universally 
approved or universally disapproved. Although Type U (describe and 
interpret a graph or chart) was the clear overall favorite^ it was not one 
of the top three choices among MBA programs and was the next-to-last 
choice for undergraduate English programs. 

Comments about writing samples . Some respondents supplied additional 
comments concerned with writing samples in the "comments" blanks on page 
6> in the margitis throughout^ or at the end of the questionnaire. Most 
of the comments about specific topic types were coded> and have been 
incorporated in the results section discussing the evaluations of the 
topic types. 
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A few respondents offered re commendations for writing sample 
topics^ ranging from personal essays to complex tasks; several comments 
supported the importance of a writing sample that would demonstrate 
students* ability to put thought processes into words* Only nineteen 
respondents from the different departments preferred a field-specific 
topic ^ such as the document at ion of procedures in c ivil engineering; 
however^ they emphasized the field-specific writing skills that might be 
elicited rather than content* With regard to cultural considerations^ 
very few respondents commented that the content^ as well as the modes of 
rhetoric required by a topic ^ should not Impose limitations on students 
f rom non-Western cultures * Approximately twenty respondents » spread 
across the departments^ also offered comments about the scoring and 
possible uses of a writing sample on TOEFL* 

Multidimensional scaling of topic types * To get a summary picture of 
the relationships among topic types both within and between academic 
disciplines^ the acceptability ratings were analyzed using a multidimen- 
sional scaling approach that accommodates differences between raters* 
Within each discipline^ the pattern of responses to each topic type was 
compared (to the pattern of responses for every other topic type* In 
mul t id im^ns ional seal ing terminology ^ these differences are called 
distancesV 3 small distance indicates a very similar response pattern 
for a pair of topic types* Pairwise topic distances were computed by 
weighting responses as shown in Table 19* 



TABLE 19 

Response Weights for Topic Distances 
Topic Y 

Good Fair Unacceptable 



Good 0 1 2 

Topic X Fair 1 0 1 

Unacceptable 2 10 



If the ratings given by a different department were the same for topics X 
and Y (e*g* ^ both topics receive good ratings)^ the response weight was 0> 
making the distance estimate 0* If the ratings differed by one rating 
category (e*g*j good vs* fair or fair vs * unacceptable) » the weight of 
1 was assigned^ and if ratings differed by "-'^o categories (good vs* 
unacceptable)^ a weight of 2 was assigned* For each pairwise comparison^ 
the sum of response weights over each department in an academic discipline 
was then computed* Next^ the sum of the weighted responses was divided by 
the number of respondents (i*e* ^ departments repoi^ting data) in the 
di s c ipl ine to get comparable numbers for disc i pi ines represented by 
unequal numbers of respondents* Separately for each discipline^ the 
distances were entered into 10 x 10 topic-by-topic distance matrices* 
These standardized distance matrices are presented in Appendix E* Lower 
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numbere on these tables indicate topics that were rated elmilarly» while 
higher numbere indicate a relatively large distance between the ratings 
(i.e. , one topic type in the pair is seen as considerably more acceptable 
than the other) . 

The 10 X 10 topic"by*^topic distance matrices for each of the seven 
disciplines were analyzed with SINDSCAL (see Carroll, 1981 » for a detailed 
explanstion of the operation of this program). Solutions in one» two» and 
three dimensions were computed. The two-dimensional solution was chosen 
for ease of interpretation and because three dimensions did not add 
substantial information. Once chosen, the two-dimensional solution was 
computed a second time to ensure that it was not an artifact of a single 
starting configuration or a local minimum. 

The two-dimensional solution yields a correlation of .71 between the 
raw distance data and the scaXed distance data, indicating . that the two* 
dimensional solution fits fairly well. Figure 2 dhows locations of the 
ten topic types in the two-dimensional stimulus space. Topic types thst 
are physically close to each other in this space were evaluated similarly 
by the questionnaire respondents. 

In contrast to factor analysis* with SINDSCAL the horizontal and 
vertical axes from can be interpreted directly without rotation* However* 
names for the axes are somewhat arbitrary, in that they are based on the 
interpretation of the relationships among the topic types as they sre 
grouped within the stimulus space. The positions of the topic types* 
as rated by the respondents » reflect their perceptions of the similarities 
and differences among the topic types. Based on the a priori influences 
we assumed when designing the contrasting topic types, and on the comments 
offered by respondents, we have labeled the axes in the stimulus space* 
The topic types on the left side of tfc> space (U, C, B) appear to be 
relatively straightforward and to require fewer analytic thinking skills; 
the desc i^^tion tasks may impose fewer information processing demands 
on the writer as the material is organized for presentati^m. The topics 
on the right vG, I, J) of the vertical axis appear to require more 
complex, or combinations of more different kinds of, thinking (hence 
organizational skills); the compare and contrast topic type (D, E), for 
example, expects students to generate two parallel lines of reasoning and 
relate them to one another. Further, extrapolation (F) requires the 
writer to extend his or her thinking beyond the data at hand, summarize d 
passage (l) implies the app3 ication of analytic skills to a teading 
passage, and sumoiarize plus analyze and assess (J) combine descriptive and 
analytic skills, as well as some degree of reasoning. The respond<nts jAiq 
selected these items, in fact, indicated preference for jtopic types thst 
also would elicit a demonstration o£ reasoning or thinking processes* 
Thus this horizontal axis was simply lab^sled ''complexity.*' 

The vertical ax^^ read from the top down, appears to represent 
the extent to which tot^.^. types demand th ,t the writer bring personal 
knowledge and experience the writing task. Most of the topic types 
that fall within the upper U-lf of the stimulus space (H, I, J) supply 
the material (graph or chart, reading passage) that the writer uses In 



Figure 2 

.Location of Topic Types in Two-Dimensional Stimulus Space 



0.0 



d.2000 



0.4000 



d.«000 



d.400t 



d.4000 



0.2000 



H^Describe & Interpret 
Graph, Chart 



c-Descript ion-Spatial 
Functional 

Comple xity 



u 
c 

> 

c 



to 
c 
o 

u 



•-Description-Sequential 
Chronological 



l-^Summarize Passage 



J*-Sunimariz« 
plus Analyze, Assess 
Point of View 



^-Arpjumentation 



-0.2000 



i*r Essay 
Personal 



r -Extrapolation 



'0.4000 



jrConpare/ 

totitrast^'^^^'^P^^^/^^^^^^^^ & Take 
Position 



H).4OC0 



-o.«ooo 



ERJC 



-0.4C00 



-0.2000 



O.d 



0.2000 



0.4000 



0.«000 



GG 



-53- 



completing the task requlreoaents ; the degree to which an individual writer 
inserts personal knowledge or experience depends on that individual's 
approach to the task. Xwo of the description tasks (C> B) are located 
near the midpoint of tMs dimension. However, topic type although 
requiring production of an image to be described spatially or functionally 
from memory^ represents a relatively concrete description task. Topic 
type on the other hand, may require somewhat more abstract mental 
processes > in that a time dimension represents the application of a 
personal framework of time and spatial sequencing. At l!ie othar extreme 
of this ^'personal involvement" dimension, the topic type stimuli suggest 
only a central theme or idea, but the information and concepts that need 
to be presented in the piece must be supplied by the writer, drawn to a 
large extent from his or her knowledge and experience. Further research 
is needed to clarify the interrelationships among these topics. ' 

Differences between disciplines are presented in Table 20. The 
entries in the table indicate the weight that respondents from each 
academic discipline placed on each dimension when judging the accept- 
ability of topic types. Thus > for exampl e > electrical engineering 
departments put great weight on the complexity dimension (other data 
suggest that they prefer topics on the less complex end of this dimen- 
sion), but relatively little weight on the personal knowledge dimension 
(i.e.> this dimension is not as important for grouping preferences of 
electrical engineering departments). 

TABLE 20 

Discipline Weight Matrix^ 

UGE HBA CE ' EE PSY C CS 

Complexity .38 .36 .A6 .85 .05 .72 .50 

( horizontal 
dimension) 

Personal Involvement .82 .17 .28 .31 .54 .27 .55 

(vertical 

dimension) ; 

^In the SINDSCAL documentation this is referred to as the subjects weight 
matrix . 



Correlations between the computed scores and scalar products for each 
academic discipline indicate how well the multidimensional scaling model 
fits the data for that discipline. The correlations are as follows; 
undergraduate English (.90) > business management (.39), civil engineering 
(.54), electrical engineering (;90), psychology (.54), chemistry (.77)^ 
computer science (.75). Thus, except for business management programs, 
the model appears to fit fairly well> and it fits particularly well for 
undergraduate English and electrical engineering. 
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Section IV. Summary and Conclusions 



A survey of the acadecilc writing skills needed by beginning under* 
graduate and^graduate students was conducted. The questionnaire was 
completed by*^[£^ulty In 190 academic departments at 34 universities In the 
United States and Canada with high foreign student enrollments. At the 
graduate level, six academic disciplines with relatively high numbers of 
nonnatlve students were surveyed: business management (HBA) , civil 
engineering, electrical engineering, psychology, chenistry, and computer 
science. l)n<*ergraduate English departments were chosen to document the 
skills needed by undergraduate students. 

The major findings are summarized as follows; 

o Although writing skill was rated as Important to success In 
graduate training , It was consistently rated as even more 
important to success after graduation. 

o Even disciplines with relatively light writing requirements 
(e.g., electrical engineering) reported that some writing is 
required of first-year students. Lab reports and brief article 
summaries ore cosimon vritlng assignments In engineering and the 
sciences. Longer research papers are commonly assigned zo under- 
graduates and to graduate students in MBA, civil engineering, and 
psychology programs. 

o Descriptive skills (e.g., describe apparatus, describe a proce- 
dure) are considered Important in engineering, computer science, 
and psychology. In contrast, skill in arguing for a particular 
position is seen as very important for undergraduates, MBA 
students, and psychology majors, but of very limited Importance in 
engineering, computer science, and chemistry. 

o Faculty members Reported that, in their evaluations of student 
writing, they rely more on discourse-^level characteristics (e.g., 
paper organization, quality of content) than on word" or senfence- 
level characteristics (e.g. , punctuation/ sp*-j.llng, sentence 
structure, vocabulary size). 

6 Dlscourse'^level writing skills of natives and nounatives are 
perceived as fairly similar, but significant differences betwen 
'ves and nonnatlves were reported for sentence- and word-level , 
ills and for overall writing. A majority of departments 
t*ortedly use the same standards for evaluating the writing of 
itive and nonnatlve students, although nearly a third of the 
departments reportedly use different standards. 

o More than 80^ of the undergraduate English and graduate business 
departments indicated that a TOEFL writing sample would be 
useful. In the other disciplines, approximately 50% of the 
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departments indicated that they would like to see a TOEFL writing 
sample. 

o If TOEFL were to include a writing sample^ most respondents 
indicated that scores on three separate £ea tures would be 
preferable to a single overall score. The primary purpose for 
the sample would be to supplemen t admissions decisions about 
borderline candidates; in addition, 81% of the uiidergradua te 
Knglish departments indicated an interest in using the writing 
sample tor course placement. 

o Among ten writing sample topic types provided, Topic H (describe 
and interpret a graph or chart), was a clear favorite among the 
engineering and science departments. However ^ this topic was 
perceived as inappropriate by a majority of the undergraduate 
English faculty. Topic G (argumentation with audience designa- 
tion) was the favorite among the VSk programs; Type E (compare and 
contrast plus take a position) was also evaluated positively by 
the MHA programs and was the ^avorite among undergraduate Englisii 
faculty. 

o A 'multidimensional scaling of the topic types suggested a two" 
dimensional space defined by a canplexity dimension and a personal 
i:ivolvement dimension. Topic H (describe and interpret a graph or 
cUart) can then be seen as a relatively simple and impersonal 
t^^sk. Topic E (compare/contrast and take a position) is a little 
above average on the complexity dimension and is a task lequiring 
a relatively high degree of personal involvement* in the topic. 

Thus» from the standpoint of the Canale and Swain framework^ the 
facul ty members surveyed appear to view the written communicative 
competencies of their students predomin^ ncly from the perspective of 
sociolinguistic competence^ placing considerably less emphas is on 
grammatical competence. For example^ the written products prepared by 
students in the different disciplines may be considered competent to the 
extent that they meet the task demands — particularly kinds o£ writing 
assignments and certain skills^-that are specific to a discipline. In 
addition, faculty members reported th«;t written assignments are evaluated 
on the basis of discourse-level characteristics, rather than word- or 
sen tence-leveJ characteris tics ^ and tlrat they perceived the discourse" 
level writing skills of natives and nonnatives to be fairly similar. 
Grammatical competency > however, ti^nds to influence evaluations of student 
writing to some extent, in that respondents reported that nonnatives are 
more deficient in word- and sentence-level skills than are natives. 
Finally, since one-third of the respondents stated that nonnatives may be 
evaluated more leniently than natives^ the strategic competencies, or 
coping strategies, of nonnative students appear to have some bearing on 
thftir success in these institutions. 

Although some important coomion elements among the different depart- 
ments were reported^ the survey data distinctly indicate that different 
disciplines do not uniformly agree on the writing task demands and on a 
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single preferred mode of diacourse for evaluating entering undergraduate 
and graduate atudents* The extent to which essays written in different 
discourse modes produce different rank orderirgs of atudents remains to 
be seen. Furthermore, if significant differences in rank ordering are 
observed, the relationship of these orderjngt; to TOEFL scorea, both 
within and acroas academic disciplines and language groups, ia yet to be 
determined. The survey results reported here are an Important beginning 
to a construct validation of TOEFL aa a measure of writing proficiency, 
but they are only a first step. 
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Appendix A 



TOEFL Research Questionnaire 
(Graduate Level Form) 



TOEFL RESEARCH QUESTIONNAIRE 



The main objective of this questionnaire is to obtain a valid description 
of the kinds of writing tasks that are required of students in your department 
during "typical" coursework. 

Since our concern is with the eocpectat ions held for entry-Ievel students, we 
are asking you to provide information that focuses on the kinds of writing that 
students are asked to produce during their first year ^f graduate schocl. Please 
do not supply Information about writing skills that your students acquire during 
their educational experiences beyond the first year* 

We assume that the writing tasks assigned to students are the same for both 
native and non-native speakers of English; therefore we ask you to respond to 
the questions in the context of all students in sections where we do not designate 
specifically that you respond from the standpoint of non-native speaking students. 
As you respond to questions about writing tasks, do not Include remedial English 
or writing courses that students take in order to meet the minimum standards of 
regular coursework* 

Your expert observations are valuable to us. 

I. BACKGROUND INFORMATION 

Institution ^ 

Department 

Humber of graduate students in your 
department who are not native 

Speakers of English 

Percentage of graduate students 

In your department who are not ^ 
native speakers of English 

Indicate the geographical area from which mojt of your 
non-native speaking graduate students come. (Check one) 

Africa Middle East 

^ Canada South and East Asia 

Europe Other (specify) 

Latin America 
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II. WRITING TASK DKMANDS 

The Eolloving questions pertain to the kinds of writing that 
f irst-^year graduace students In your department are expected 
to produce in all of their courses. 

For each of the following writing tasks, indicate how frequently 
each task might be assigned to students per semester . (Circle 
one number for each task,) 



WritinR Tasks 

1, Lab reports or descriptions of 
experljnents conducted by the 
student or in class 

2, Brief s'jmmari^is of arcicles 
read (one or cwo pages) 

3, Brief research papers 
(5 pages or less) 

A, Longer research papers 
(6 pages or more) 

'5, Creative writing (fiction, 
poetry, or drama) 

6, Expository or critical 
writing unrelated to lab 
or library research 



7, Ex^ms with essay questions 

8. Group writing projects 

9- Case studies 

Other (specify) 



10 



Not 
at all 



11. Other (specify) 



0 
0 
0 
0 
0 
0 



1 or 2 3 Co 6 7 or more 
tljnes per tljnes per times per 
semester semester semester 



2 
2 
2 
2 
2 
2 



3 
3 
3 
3 
3 
3 
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Indlcate the degree of Importance for each of the following 
items. (Circle one number for each.) 



Degree of Importance 



1. How Important is writing skill to 
success in your department? 

2. How Important is writing 'skill to 
success in your field after 
graduat ion? 



Indicate the importance of -each writing skill 
for success in the firat year of graduate study; 



Low Moderate 
12 3 4 



3. Describing an object or apparatus 

4. Describing a procedure 

5. Arguing persuasively for a 
particular position 

6. Organizing arguments drawing on 
several different sources 

7. Summarizing factual Information 
from a single source 

1, 

8. Analyzing or criticizing ideas, 
excerpts* or passages 

9. Expressing oneself creatively 

10. Other (specify) 

11. Other (specify) 



2 
2 

2 

2 

2 

2 
2 
2 

2. 



3 
3 

3 

3 

3 

3 

3 
3 
3 



4 
4 



4 
4 
4 
4 



High 
5 



5 
5 
5 
5 
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III. CRITERIA USED TO EVALUATE WRITTEN ASSIGNMENTS 



The following Is a list of features of written assignments. 
After first reviewing the full list, conslde.. the extent tc 
which grades for written assignments In your courses for 
beginning graduate students are Influenced by each feature. 
By circling the appropriate number, rate each feature for the 
degree of. Importance you place on It. 



General Features: 

1. Correctness of punctuation/ spelling 

2. Quslity of sentence structure 

3. Size of vocabulary 

A. Appropriateness of vocabulary usage 

5. Quality of paragraph organization 

6* Quality of overall paper organization 

.7. Quality of content 

8. Dev^opnent of Ideas 

'9. Overall writing ability 



Degree of Importance 



Low 



Moderate 

3 
3 
3 
3 
3 
3 
3 
3 
3 



2 
2 
2 
2 
2 
2 
2 
2 
2 



4 
4 
4 
4 
4 
4 
4 
4 
4 



High 

5 
5 
5 
5 
5 
5 
5 
5 
5 



fleeting constraints of psrticular assignments: 

10. Student addresses topic adequately 

and directly 1 

11. Student adopts a tone^ attitude, 

or style appropriate to the audience 1 

12. Student appropriately meets assignment 
requirements as specified by Instructor ^ 

13. Other (Specify) 1 



14. 



3 
3 
3 



4 
4 
4 



5 

5 

5 
5 
5 



15. Which of the features (1-'X4) listed above are most and least Important? 

(Specify one number from the above list for each) 

Most important Least Important 

16. Do you u'ie the same standards to evaluate the writing o2 natxvt: and 
non-native speakers of English? 



YES 



NO 



If NO, please circle the numbers (from the list above) of the 
features you evaluate more leniently for non-native speaking students: 

1 2 3 4 5 6 7 8 9 10 11 12 13 14 



ERIC 



So 



-5- 



IV. WRITING PROBLEMS OF NATIVE AND NON-NATIVE SPEflJCING GRADUATE STUDENTS 



This section asks you to make two judgments about each of the 
features of written Assignments: 



1. 
2. 
A 

Problems of Native 
Speaking Students 

Minor or Rare 



In column A^ indicate the extent to which each feature 
is a problem for native speakers of English. (Circle) 

In column B, indicate the extent to which each feature 
is a problem for non-native _speakers of English. (Circle) 

B 

Problems of Non-ilative 
<^tifiakint; Students 

Minor or Rare 

Moderate or Occasional 





Moderate or 


Occasional 




i 


Large or Frequent . , „ 
1 ^ General Features: 


1 


2 


3 


1. 


Correctness of punctuation/spelling 


1 


2 


3 


2. 


Quality o^ sentence structure 


1 


2 


3 • 


3. 


Size of vocabulary 


1 


2 


3 


4. 


Appropriateness of vocabulary usage 


1 


2 


3 


5. 


Quality of paragraph organization 


I 


2 


3 


6. 


Quality of overall paper organization 


1 


2 


3 


7. 


Quality of content 


1 


2 


3 


8. 


Development of ideas 


1 


2 


3 


9. 


Overall writing ability 



1 1 



Large or Frequent 

i ' 

3 
3 
3 
3 
3 
3 
3 
3 
3 



1 2 3 

12 3 

12 3 

1 2 3 

12 3 



Meeting contraints of particular assignments 
10 



11. 
12. 

13. 
14. 



Student addresses topic adequately 
and directly 

Student adopts a tone, attitude, or 
style appropriate to the audience 

Student appropriately meets 
assignment requirements as 
specified by instructor 

Other (Specify) 



2 3 

2 3 

2 3 

2 3 

2 3 
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A WRITING SAMPLE IN THE ADMISSIONS PROCESS 



1, Does your institution administer a writing sampl e at the time 
of admissions? 



YES 



NO 



(Skip to Question 3 if your answer is NO) 



2. If your institution uses a witing sample, for what purpose ? 
(Check as many as apply) 



Purposes 

as an essential admissions criterion 

for admissions decisions on borderline cases 

for plact*3»Gnt in coucses 

for cajnpus job placement * 

other (specify) 



For Native Speakiny? Students 



Foi Non^-Nati/e 
Speaking Students 



3. Beyond the admissions criteria yc \re now using, would any other 
additional information about non-na tive speaking applicants be 
helpful to your department or institution? 

YES NO ' (If YES, specify): 



Would a writing sample on TOEFL be useful to you? 
YES NO 

5, I^ a writing sample were included on TOEFL, for what purpos es would 
ypu use it? (check as many as apply) 



as an essential admissions criterion 

for admissions decisions on borderline cases 

for placement in courses 

for campus job placement 



other (specify) 



6, How would you prefer to have the TOEFL Wi^Xting sample scored ? (check one) 

^ single score for overall impression of writing ability 

separate scor^iS for: 

a, content, quality of ideas 

b, grammatical/meoiianical errors 

c, organization and coherence in the writing 

Additional comments you would Hke to make:- 
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VI. EXAMPLES OF TOPICS FOR WRITING SAMPLES 

The following is a list of examples of possible topics for a writing 
sample for non-native speaking applicants (for use in admissions or 
placement, at the beginning of graduate work). Before you indicate 
your preferences for topics, think carefully about: 

1. the information that you want to'^ obtain from the sample, and 

2, whether or not the topic will evoke a writing sample that 
will provide that, informatior.. 

The following pages contain two examples for each of several types of 
topics that might be used to elicit writing samples* labels are supplied for 
the topic types ^n order to help you categorize them; they do not imply a 
formal taxonomy, but have been provided as descriptions. Please respond to 
the type of topic rather than to the specific examples. 

For each of the following types of topics, you will be asked to indicate 
its degree of acceptability for a topic for a writing sample that inight .be used 
as part of the admissions process for beginning graduate students. At the end 
ot the presentation of topic types, you will be asked to indicate the types you 
think are most appropriate. 
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Types of Topics to Elicit Writing Samples 

Tytj e A. EssaV — Personal 

# Pt'Oplc choos€* thc^lr academic majors (fields of study) for a variety of 
reasons, De?5crib<ir your re*isons for choosirtg your academic major. 

• Pf?op] f? always find faglL with the other generation, whether older or 
younger.* Wliat are some characteristicc that you admire most about your 
genf^rat ion? 

Que^>t£ofp^ of thl^ '-yp*' arer Acceptable/good, Acceptable/fair, Unacceptable. 

(circle one) 

If unarc<-ptablej t^xplain why: 



-XyP*^> .Hes c'^ ^P^ . ^^ - P — Sequen t ial or Chronological 

♦ Another student asks for your advice on^'how to study for a big test. 

Describe a step-by-step plan that you would recommend to him/her, begin- 
ning a week before* the test. 

« Choose a job you ^,id in the pas^^* Select one or two of the most important 
tasks you performed in this job. For each task^ describe the order in 
which you performed the duties required to accomplish the task durin^j a 
typical day (or week). 

Questions of this typ*:; are: Acceptable/good, Acceptable/fair. Unacceptable. 

(circle one) 

If unacceptable, explain why: 





Type C. D escription — Spatia l or Functional 

« Think of a toy that you or someone you knew enjoyed as a child. How would 
you describe It? Briefly explain how the toy was used in play. 

♦ Someone you know has never seen a school or public library. Describe such 
a library> and briefly explain how it be used. 

Que<;rions of this type are: Acceptable/good, Acceptable/fair, Unacceptable. 

(circle one) 

If unacceptable> explain why: . 
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rypt*s of Topics to Elicit Writing Samples (cont J 
Type D, Compare/Concrast 

• Advances in technology bring advantages and disadvantages. How does life 
in a highly technological society differ from life a soc iaty that ha& 
little technology? Select one or two advantagei* as well as one or two 
disadvantages that you liave observed, and explain them. 

• Being a student in a foreign country has its advantages and disadvantages. 
Sele^L one advantage and one disadvantage of studying in a foreign country, 
anci discuss them briefly. 

Questicns of this type are; Acceptable/good, Acceptable/fair, Unacceptable* 

(circle one) 

If unacceptable, explain why; 



Type E. QpRipare/Contrast Plus Take a Position 

• Living in a country with advanced technology has its advantages; however, 
life in a country with little technology also has unique characteristics. 
Describe one or two of the advantages of living in a country with advanced 
technology, as well as one or txjo of the advantages of living in a country 
with little technology* Decide in which country you would prefer to live,, 
and briefly explain your reasons, 

• In their choice of professions, some people prefer to work with other people, 
whereas some people prefer to work by themselves. Describe one or two of 

the advantages of working with others, as well as one or two of the advantages 
of working alone. Decide whether you would prefer working with others or 
alone during most of your work day, and brief-.,y explain your reasons. 



Questions of this type are: Acceptable/good > Acceptable/fair, Unacceptable* 

(c ircle one) 

If unacceptable, explain why: 
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Types of Topics to Elicit Writing Samples (cont.) 



T ype F. Extrapolation 

? You have been assigned to the task of choosing three objects to be placed in 
a time capsule which will be buried today and unearthed one hundred years 
from today. Your task is to choose Items that represent the life of people 
in your aati/e country in the 1980*s. When these objects are unearthed, the 
people who find them should get from them a feeling of what life i? like in 
your country today. Give careful thought to your choices, and in a well- 
organized essay, describe ioc name) the objects you have chosen and explain 
your reasons for choosing each of them. 

# Everyone occasionally thinks about what life would be like in our children's 
or grandchildren*s time. What kind of world would yoa like for your grand- 
children? Describe three characteristics of the worlu you would liVe for 
them to live in, and explain your reasons for choosing each characteristic. 

\ 

Questions of this type are: Acceptable/good, Acceptable/ fair, Unacceptable. 

(circle one) 

If unacceptable, explain why: 



Type G. Argumentation with Audience Designation 

• Iiragine that a country with limited coal and oil resources is thinking of 
Installing two or three nuclaar power plants. State the arguments that are 
either in favor of, or against, the building of the plants. Direct your 
arguments to an audience that is a government committee composed of Indivi- 
duals who do not have any technical expertise. 

• Two routes for a highway are being considered. Route A would be four miles 
shorter than Route B, and would go through thr. center of town and provide 
better access to the downtown area. However, Route A would destroy many 
people*s homes. Route B would go through parkland that contains a wildlife 
sanctuary at the edge of town* Both choices are undesirable, but imagine 
that you have chosen to argue for one of the two routes. Choose the route 
that you prefer, and write your argument for presentation to the town council 
that is responsible for making the decision. If you choose Route A, imagine 
that you work for a company in the downtown area that wants Route A built. 

If you choose Route B, imagine that you belong to a homeowners* organization 
that opposes Route A. 

Questions of this type are: Accaptable/good, Acceptable/fair , Unacceptable. 

(circle one) 

if unacceptable^ explain why: 
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Type H. Describe and Interpret a Graphs Chart, etc . 

• The bar graph shows the numbers of automobile parts produced by company A, 
company B, and company C in three different years. What does this graph 
tell you? Write a few sentences that convey the Information In the graph. 



S.30 



° 25 
S 20 



15 



c 

o 



o 
p 



10 



Company A 
Company & 
Company C 



I 



i 




1950 



1960 



1970 



The pie chart shows how one family spends their annual income of $12^600, 
Each part of the chart gives the percentage spent by this family on a 
major expanse item. What does this ch^rt tall you? Write a few sentences 
that convay the Information In the ^liart. 



Other 



Med leal 
Care ► 



Transportation 




Food 



Housing 



Questions of this type are: Acceptable/good, Acceptable/fair, Unacceptable. 

(c ircle one) 

If unacceptable^ explain whyi , 
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Jyj ^ ^' Stjmmarize a Passa ge 

Read ttie following passage to understand thfi ideas in it, even though you 
may not agree with the point of view expressed by the author. 

In one century, we liave come to knov' more about the sea 
than we learned during all preceding history. We now under- 
stand that the ocean influencea the broad cycles -yf cllniaLf* 
and weather, that it absorbs most of the sunlight that strikes 
the earth and creates most of the oxygen we breathe, that it 
absorbs huge amounts of carbon dioxide, and that it is the 
source of most fresh water. 

We have also learned that in the great spans of time 
measured by geology we are relative nex«:omers. However, we 
are causing in our lifetimes changes as far-reaching as 
those effected by the geological events of the ages that 
prec?,ded us. 

We join by canals waters that have been separated for 
millions of years. We create dry land out of seafloor and 
carve harbors from marshlands that were tens of thousands 
of years in the making, in a matter of just a few decades, 
we have inadvertently spread films of oil and chemicals upon 
the very surface that supplies life-giving oxygen and fresh 
water . 

We also know that we have set in motion cycles that may 
drastically alttr the ocean as we know it today. The ocean 
realm is vast and diverse — but also fragile. Oil spills, over- 
fishing, dumping of garbage, chemicals, and radioactive sub- 
stances tiave all diminished the health of the ocean. But much 
of the wilderness character of a century ago remains, and with 
care and an understanding of the value of the sea, that character 
can be preserved. 

One of the most significant things learned in this 
century, I believe, is the importance of the sea to our well- 
being and survival, "he necessity of an ocean wilderness to 
the health of the world's environment. We have learned that 
the ocean greatly affects all living things, f^om the deep-sea 
fish to the desert mouse. 

Write a brief summary of the preceding passage, describing the main ideas. 
Use no more than five sentences to show that you understand the main ideas of 
the passage. 



So 
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Read the following passage to understand the ideas in lt> even though you 
may not agree with the point of view expressed by the author. 

If we are not crazy^ and we wiJl assume we are not^ why 
Is It that humanity seems determined to spiral ever faster towards 
self-made destruction? Perhaps the human species is Just a 
ghastly biological blunder^ having evolved beyond a point at which 
it can thriAje in harmony with itself and the world around it* 
That must be a possibilltv. In recent years scientists^ play- 
wri£hts> and oth£*rs have attempted to explain why mankind finds 
itsolf faced with the prospect of self-destruction. The idea was 
proposed that man is unswervingly ag(^ressive> an idea that was 
given scientific credence by proponents such as Professor Raytnond 
Dart and Dr* Konrad Lorenz^ and euccessfully popularized by 
Rob^&rt Ardrey* 

The core of the aggression argument sayf- that because 
we share a comnwn. heritage with the animal kingdom we must 
possess and express an aggressive instinct* And the notion 
is elaborated with the suggestion that at some point in our 
evolutionary history we gave up being vegetarian ape-like 
creatures and became killers^ with a taste noL only for pi^^^y 
animals but also for each ether. It makes a good gripping 
story, }^oxe Important^ it absolves society from attempting 
to rectify the evil in the world. But it is fiction — dan- 
gerous fiction. 

Unquestionably we are part of the animal kingdom* And > 
yeSy at some point in our evolution we departed from the 
common dietary !iabits of the large primates and took to 
including a significant amount ol meat in our menu. But a 
serious biological interpretation of these facts does not lead, 
to the conclusion that> because once the whole of the human 
race indulged in hunting as part of its way of life> killing 
is in our genes. Indeed^ I would argue that the opposite is true> 
that humans could not have evolved in the remarkable way in 
which we undoubtedly have unless our ancestors were strongly 
cooperative creatures. The key to the transformation of a 
social ape-like creature into a cultural animal living in a 
highly structured and organized society is sltaring; the 
sharing of jobs and the sharing of food. Meat eating was 
Important in propelling our ancestors along the road to 
hiimanity^ but only a^i part of a package of socially-oriented 
changes involving the gathering of plant foods and sharing 
the spoils* 



This being so, why then is recent human history charac- 
terized by conflict rather than compassion? i sufifiest that 
the answer to this question lies in the change in way of life 
from hunting and gathering to fanning, a change which began 
about ten thousand years ago and which involved a dramatic 
alteration in the relationship people had both with the world 
around them and among themselves. The hunter-gatherer is a 
part of the natural order; a farmer necessarily distorts that 
order. But more important, stable farming communities have 
the opportunity to accumulate possessions, and having done so 
they must protect them. This is the key to human conflict, 
conflict that is g^^tly exaggerated in the highly mater- 
ialistic world in wh'ich we now live. 

Write a brief summary of the preceding passage, describing the main ideas. 
Use no n.ore than five sentences to show that you understand the main ideas 
of the passage. 



Questions of this type are: Acceptable/good, Acceptable/fair, Unacceptable, 

(circle one) 

If unacceptable, explain why; 
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Type J> S ummariz e a ? ;assagr Plns Analy^^ti/AsscHS the Point of V j-_ew 

o Re;id the following passa;;e to ijnderstand the ideas in it, even though 
yOK} Diay not agree with the int of view expressed by the author. 

(H(-rf Lhe 5;tiident wmld he prosc^nted vith the sam^' 
passd^;e an in Typt> I, beg^lnning with, "If we arc 
not cra^v. - . .**) 

Urite a summary of the preceding passage, describing the main ideas. 
Vsi^ no more than five Sf*ntcnr<?s to show that you understand the main 
ideas of tht> passage. 

Uht*n you hnve completed /our summaryj write a brief analysis of 
the ide^^s in the passage. You may Include your own ideas as well 
as ctose you ind in the passage. 

• Read tht following passage to understand the ideas in it, even though 
you may not agree with the point of view expressed by the author. 

Intimately related to the energy issue is size of 
population. There is a simple equation that says that, 

with limited resources to exploit, the life span of modern ^ 
human society is inversely related to the number of people 
in that society. In other words, the more people there are, 
the sooner resources are used up, With the exception of the 
sun's rays, there is no resource that is not limited in some 
important way. True, future technologies are certain to 
exploit materials in ways not dreamed of at the moment. And 
it is possible that new technologies could guide the human 
species through original material aSSieties for many thousands of 
years. But the kev to successful ®cploitation of new material 
resources is trtat the process should be unhurried, so that * 
the maximum potential sho*jld be squeezed out of the material 
world. With the population due to double in the next thirty 
years, and to double again in less than thirty years after that, 
there is no prospect of such an unhurried approach. Who can 
stand up and confidently say that the rate of innovation 
will match the rocketing demand? Currently the gap is 
large, and it is widening minute by minute. 

One obvious solution to the problem of a large world V 
popuUtion is to have a ^mall one instead. It is quite 

reasonable to propose tliat instead of the current four thousand \ 
million, a population of half that would be appropriate for \ 
the lon^-tetm survival of a well-fed world, population. Such 
a proi^pect is a very long way off, even if every country agreed 
on Iti good sense. But, as the 1974 World Population Conference 
demonstrated all too clearly, global control of population ia 
fraught vith even more snares than i'; the problem of energy. 
Quite apart from the serial and cultural barriers that continue 
to thwart national population plans, international politics 
co^ild be as much a barrier to global population control as 
a vehicle for its adoption. 

. 
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For instance, at the 197A conference in Bucharest, pro- 
porsals by the rich nations for cutting down the rate of 
population growth were interpreted by a number of the poorer 
nations as merely another facet of iiuperialism-^popijlatlon 
imperialism. Who can blame them? Experience has taught these 
nations to be wary of the intentions of the affluent world. 
Unless the accumulated layers of suspicion that separate 
nations of different economic status and different political 
ideology can be cut through and stripped away, however, the 
path for global decision-inaking will remain blocked. And 
if that path does remain blocked, the threat to the con- 
tinued survival of mankind becomes very real indeed. 



Write a summary of the preceditig passage, describing phe main ideas. 
Use no more than five sentences to show that you understand the main 
ideas of the passage^ 

When you have completed your summary, write a brief analysis of the 
ideas in the passage. You may include your own ideas as well as 
those you find in the passage. 



Questions of this type are; Acceptable/good, Acceptable/ fair, Unacceptable. 

(circle one) 

If^unacceptabLe, explain why: 



* * * * * * ^ * 



As you review the entire list of sample types, which types of topics are 
most appropriate? Using the letters from the preceding list, list up to five 

preferred types, in order, starting with the most appropriate; 

Please list any additional topic types that you would like co recommend. 



Comments: 



Thank you very Tiuch for your efforts and information! 
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Enrollment Data for Contributing Institutions 
in the Research Sample^ 



Region 



Northeast 



Midwest 



Pacific 



Institution 



Private/ 
Public 



No. of 

Foreign 

Students 



% of 
Foreign 
Students 



Total En- 
rollment 



Boston University 


1 


2,015 


7.2 


27,796 


Columbia University 


. 1* 


2,591 


10.9 


23,741 


Harvard University 


1 


1,645 


10.4 


15,821 


Northeastern University 


1 


2,081 
1,749 


5.0 


41,343 


SUNY, Buffalo 


2 


8.0 


21,759 


University of Pennsylvania 


1 


1,686 


7.5 


22,611 



Indiana University 
Ohio State University 
Purdue University 
Southern Illinois University 
University of Illinois 
University of Kansas 
University of Michigan 
University of Minnesota 
University of Wisconsin ' 

California State U. , Fresno 
Oregon State University 
San Francisco State University 
Stanford University 
University of Califoniia> 

^os Angeles 
University of Hawaii* Hanoa 
University of San Francisco 
University of Southern 
California 



2 


1,772 


5.6 


31,877 


2 


1,832 


3.4 


54,533 


2 


1,196 


3.6 


32,978 


2 


1,566 


6.7 


23,236 


2 


1,521 


4.4 


34,791 


2 


1,585 


6.5 


24,465 


2 


2,104 


5.8 


36,311 


2 


1,891 


2.7 


68,907 


2% 


2,280 


5.5 


41,349 


2 


1,163 


7.8 


14,911 


2 


1,053 


6.0 


17,682 


2 


982 


c 


c 


1 


1,502 


11.1 


13,592 


2 


1,990 


5.8 


34 ,023 


2 


1,102 


5.4 


20,319 


1 


1,463 


24,0 


6,086 


1 


3,456 


12.6 


27,471 



South/ 
Southwest 



Florida International 
University 

George Washington University 
Harvard University 
University of Houston 
University of Hiami 
University of Oklahoma 
University of Southwest 
Louisiana 

Univt sity of Texas, Austin 



1 
1 
1 
1 

2 
2 



1,112 


9.5 


11,673 


1,898 


9.1 


20,844 


1,541 


13.6 


11,321 


1,789 


5.8 


30,693^ 


1,803 


11.5 


15,715 


1,611 


7.5 


21,615 


1,535 


11.1 


13,853 


1,880 


4,1 

J 


46,148 


an, 1981); 


data 


for three 
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Canadian institutions (McGill, U. of Toronto, U. of Waterloo) were not available 
in this publicstion (locations are on main campuses unless otherwise noted) 

^1 » Private, 2 » Public 

^Percent foreign and total enrollment figures not reported . 
in Open Doors: 19Sn/ftl. ^ 
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Coordinators for the TOEFL Research Study 



Coordinators - TOEFL Research Study 



Academic Writing Skills and Tasks 

South/Southwest 



Pacific 

Mrs* Carol bfunshower 
International Student Counseling Office 
California State University * Fresno 

Ms. Ann Larson 

English Language Institute 

Oregon State University 

Dr« Harry Freeman, Coordinator 

Office of International Student Programs 

San Francisco State University 

Ann Fletcher 

Office of Graduate Admissions 
Stanford University 

Dr« Carol Hartzog 

UCLA Writing Program 

University of California * Los Angeles 

Mrs* Joyce Settle 
Arts & Sciences 

Student Services and Special Programs 
University of Hawaii 

Professor Phil Carleton 
World English Center 
University of San Francisco 

Ms. Mary Berg 

Acting Director, International Admissions 
University of Southern California 



Mr* John A« Bonanno 

Director^ International Student Services 
Florida International University 

Dr. Patricia J. McMillen 
International Services 
George Washington University 

Mrs* Adrienne W« Price 
Director of Admissions 
Howard University 

Dr« Jack Burke 

Director, International Student Services 
University of Houston 

Ms. Laura L. Morgan 
International Student Services 
University of Miami 

Dr. Eddie Smith 
Dean of Admissions 
University of Oklahoma 

Mr. Bruno E. Masotti 
Director, International Office 
University of S.W. Louisiana 

Dr. Joe W. Heal 

Director, International Office 
University of Texas, Austin 
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Coordinators - TOE?L Research Study 
Academic Writing Skills and Tisksks 



Canada 

Ms* Peggy Sheppard 
Director, Admissions Office 
McGill University 

Ms* Joan Hjlven^ 

Associate Registrar of Admissions 

University of Toronto 

Ms. Helen Ben Susan 
Associate Registrar 
University of Waterloo, Canada 
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Coordinators - TOEFL Research Study 
Acadanic Writing Skills and Tasks 



Northeast 

Mr. Theodore Dlef f enbacher 
International Student Office 
Boston University 

Dr. Louis L^l 

American Language Program 

Columbia University 

Mr. C. John Frlesman 

Admissions Office 

Graduate School of Arts £t Sciences 

Harvard University 



Dean Paul Krueger 
English Language Center 
Northeastern University 

Dr. Stephen C. Dunnett 

Director, Intensive English Language Inst. 
S^m at Buffalo 

Mr. Clay Naff 

Office of International Programs 
University of Pennsylvania 



Midwest 
Mr. Marlln Hovard 

Center for English Language Training 
Indiana University 

Dr. Jules Lapldus 

Graduate School 

Ohio State University 

Mrs. Margery Ismail 

Director of International Student Services 
Purdue University 

Ms. Beverly Walker 
International Services 
Southern Illinois University 

Miss Lydla Salonga 
Admissions Processing Center 
University of Illinois 

Dr. Clark Coan 
Foreign Student Office 
University of Kansas 

Dr. Dan Douglas 
Director of Testing 
English Language Institute 
University of Michigan 



Dr. Michael Paige 
International Student Office 
University of Minnesota 



Dr. Robert M. Bock 
Dean» Graduate School 
University of Wisconsin (Madison) 
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Cover Letters to Coordinators 
and Faculty Respondents 
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EDUCATIONAL TESTING SERVICE 




PRINCETON, N.X 08641 



CABlt'lDUClESlSvC 
* 

DIVISION OF FJWCATIONAl 
RESEARCH AND EVALUATION 



October 12, 1982 



* 



Thank you for your enthusiastic response to the TOEFL Survey 
of Academic Writing Tasks and Skills* As the coordinator at your 
institution, you are vital to the success of this research project. 

This package includes cover letters and questionnaires for you 
to distribute to the faculty respondents you select, as well as 
your copy of these materials. For your information, the white 
version of the questionnaire is designed for distribution to one 
faculty respondent who teaches regular freshman undergraduate English 
(or literature). The grey version of the questionnaire is designed 
for faculty respondents from the several graduate departtnents we 
have selected for this sample* You may find that a review of the 
questions will aid you in selecting the faculty respondents* 

As the coordinator, please choose one faculty member respondent 
from each of the following graduate departments: business (MBA 
level), chemistry, psychology, cociputer science, electrical engineering, 
and civil engineering. Also select one faculty member respondent 
from the regular undergraduate English (or literature) department. 
If your institution does not have all of these departments, please 
do not substitute a different department. For this preliminary 
study> our sample has been reatricted to these departments because 
they represent fields i\\ which a large proportion of international 
students in the U. S. and Canada are enrolled. When you return the 
completed questionnairi^s to us> please add a note to the package 
that indicates which departments are not at your institution. 



* The salutation and mailing addreas were personalized for each 
coordinator. 



100 



* 2 - 



The primary objective of this research is to identify and 
describe the various writing task demands that are necessary for 
successful academic work during the first year of undergraduate and 
graduate study for non-native speakers of English. In addition, we 
are asking whether or not the writing tasks and skills that are 
required of non-native speakers are parallel to the writing task 
demands that are expected of native speakers of English. We also 
are seeking to determine whether faculty think that the inclusion 
of a writing sample on the TOEFL examination would serve as a useful 
supplement to admissions (and placement) information about incoming 
international students* Finally, some of the questions on the 
survey instrument focus on the appropriateness of the xinds of 
writing tasks, or **topic types,'* rhat might be used as a writing 
sample in the admissions process* Depending on the results of this 
survey, we may conduct further research on the parameters, scoring, 
cultural considerations, and implementation of a writing sample for 
non**native speakers of English. 

We are relying on your judgment in selecting faculty members 
who are especially familiar with the academic writing tasks and 
skills that your institution requires of both native and non-native 
speakers of English. Your identification of these experienced 
faculty members will enable us to obtain more valid^ high quality 
data that will be used to determine further research and program 
directions. In our reporting of results, you can be assured that 
the names of individuals in your institution will not be identified 
with your data, although you and your institution will be acknowledged 
in our report . 

A few details-^-a postage piepaid, self --addressed mailing envelope 
is enclosed in this package for your convenience* If you should 
need to ship the materials by some form of express mall, please let 
us know the charges, and we will reimburse you* Please return all 
materials by November 1 > 

The last page of each questionnaire is a form that each respondent 
should complete in order to obtain the $25 payment. We request a 
social security number from U* S< respondents, not for tax reporting, 
but because it is required by our auditors. As coordinator^ please 
remove this last page of each questionnaire to preserve the respondent's 
anonymity, and return these pages together with the fortn for your 
payment in the same package. Unfortunately, due to mailing and 
procedural delays, the checks may take a month to arrive-^don * t 
anticipate them any sooner. 




If you have any comments or questions* feel free to call us 
collect at (609) 921*9000, asking for the extension we are providing 
with our signatures. We also will welcome calls from your colleagues 
whom you have selected as respondents* 

Your contribution to the svccess of this study is o( enormous 
value to us, and we very much appreciate your support. 

Sincerely, 



Bren^ Bridgeman 
Extension 5767 



Sybil Carlson 
Extension 5615 



Roberta Kline 
Extension 5782 



BB:SC:RK:jaf 
Enclosures 
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EDUCATIONAL TESTING SERVICE (^^l PRINCETON. N,J, 08541 



6og^gjt~gooo 

CABL E' EDUC TtS JS VC 
* 

DIVISION OF EIXJCATIONAI 
JtE5£AKCH AND EVALUATION 



October 12 ^ 1982 



Dear Colleague, 

We very much appreciate your wllllngaess to respond to the TOEFL Survey of 
Academic Writing Tasks and Skills. The informacion you provide will be of great 
value to tis* 

The Test of English as a Foreign Language (TOEFL) is an examination designed 
to measure the English proficiency of people whose native language is not English. 
The TOEFL is administered on regularly scheduled dates at te^t centers established 
throughout the world by Educational Testing Service (ETS) , More than 2,000 
colleges and universities lU the United States and Canada, as well as in other 
countries where English is the language of instruction, require their applicants 
who are not native speakers of English to submit scores obtained tmder one of 
these programs. In addition, many government agencies, scholarship programs, 
and other institutions use the test. Each institution or agency that uses TOEFL 
scores decides for itself what scores are acceptable* The TOEFL office does not 
determine passing or failing scores. 

The TOEFL program conducts an ongoing research program related to the test, 
with the objectives of maintaining instruments that are of high quality and of 
improving the tests and services offered to our users. We have selected your 
institution and approximately 35 other institutions In the United States and 
Canada for inclusion in our sample because they enroll a substantial proportion 
of non-English speaking students. We also have selected several specific 
departments for our sample, including yours, as departments that typically enroll 
large numbers of these students. You have been contacted by a campus coordinator, 
who has been instrumental in selecting one faculty member from each department. 
You have been identified as a faculty member who is especially familiar with the 
writing tasks and skills that your Institution requires of both native and non*- 
native speakers of English. Your responses, based on your extensive experience, 
will enable us to obtain valid, high quality data that will be used to determine 
further research and program directions. In the reporting of restJ.ts, you can 
be assured that your name will not be identified with your data, although you and 
your institution will be acknowledged in o>ir report. 

The primary objective of this research Is to identify and describe the 
various writing task denands that are necessary for successful academic work during 
the first year of undergraduate and graduate study for non^ative speakers of 
English. In addition, we are asking whether Or not the writing tasks and skills 
that are required of non-native speakers are parallel to the writing task demands 
that are expected of native speakers of English. We also are seeking to determine 
whether the inclusion of a writing sample on the TOEFL examination would serve as 
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a useful supplement jo admissions (and placement) information about incoming 
international students* Finally, some of the questions on the survey instrument 
focus on the appropriateness of the kinds of writing tasks, or "topic types," 
that might be used as ^ writing sample in the admissions process. Depending on 
the results of this study, we may conduct further research on the parameters, 
scoring, culturrl considerations, and implem'jntation of a writing san^le for non- 
native speakers of English* 

We need to meet a tight deadline for summarizing and reporting this survey 
data; therefore we would appreciate your completing and returning the questionnaire 
to your coordinator as promptly as possible* Your institution's coordinator has 
been asked to return the questionnaire by November 1 * For your efforts in respond*- 
Ing to the questionnaire, we will be paying you a $25 honorarium* The form on the 
last page of the questionnaire will be used for processing your payment* We 
request a social security number, not for tax reporting, but because it is 
required by our auditors* Your coordinator will remove tills page before remitting 
it to us, in order to preserve the anonymity of the information you supply on the 
questionnaire* Unfortunately, due to mailing and procedural delays^ the checks 
may take a month to arrive— don't anticipate them any sooner. 

The last page of this letter Includes a form that requests information about 
the sdmlsslons criteria that are used by your department* Please supply this 
information if you are familiar with your department's admissions process; if you 
do not have this information, simply leave this form blank, since this data is 
useful, but not critical, to the survey* 

If you have any comments or questions, please feel free to call us collect 
at (609) 921-9000, asking for the extension we are providing with our signatures* 

Your contribution to the success of this study is of enormous value to us, 
and we very much appreciate your efforts* 



Sincerely, 



Brent Brldgeman 
Extension 5767 



Sybll Carlson 
Extension 5615 



Koberta Kline 
Extension 5782 
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Appendix E 



Distance Matrices by Disciplines 



DISTANCB KATRIXt ENGLISH 





A 


B 


c 


D 


B 


r 


G 


H 


I 


J 






?«?f42 






a*7iM 


:i*ii25 


a*1411 


l*v911 


t*t(i?l 


i*o:i70 


d*4259 




9*7742 




0*3541 


0*7419 


0*??42 


0*?C9? 


0*??42 


0*4259 


0*1077 


t*031l 




t«906] 


0«1S4« 


0*0 


c*j?5a 


0*9061 


0*9115 


0*S1?5 


0*60?t 


0*1771 


t*074t 




0*1 M 




o*a?5b 


t'*0 


C*2$ll 


0*6?5J 


1*0625 


t*2l5? 


1*2443 


0*4630 




0*6125 


J«??42 


C*Wtl 


0*2111 


0*0 


0*5911 


0*0411 


t*205? 


1*2963 


0*9254 






0«?0«7 


0*9175 


0*6250 


C*541i 


0*0 


0*61?5 


t*t7l6 


t*0170 


0*15t4 


a 




0*1742 


0*41?^ 


i*0425 


0*84* 


:i*6i?5 


0*0 


a*7157 


0*4246 


0*140? 


H 


taoit 




0*t07t 


t*2ll5? 


1*2157 


t.l?16 


0*115? 


0*0 


0*6S»6 


1*0000 


1 




0«U?7 


o*7??a 


t*2961 


t*2961 


t*01?0 


0*62 «6 


0*4244 


0*0 


0*4000 


i 




t«01iS 


t*C?4t 


0*9610 


0*9254 


0*0514 


0*?40? 


t*0000 


0*4000 


C*0 
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DISTAHCB HATRIX: MBA (BUSINESS) 





A 


B 


C 


0 


E 


F 


C 


H 


I 


J 


k 


0.0 


O.IUl 


0.62U 


0. ?»00 


0.96^1 


0.6?6e 


0.6929 


0.16^2 


0.(611 


0.8616 


0 


O.IUl 


u.O 


0.60?t 


0.»}5? 




C.(}(? 


0.6166 


0.(165 


0.1061 


0.7127 


c 


0.9ZU 


0.60?t 


O.C 


0.?6»? 


t.CIU 


0.6^29 


t.OlU 


0.1691 


0.6250 


0.9091 


0 


O.I^OO 


0.»}»? 


0.?6»? 


0.0 


0.5000 


0.6^29 


0.(0G0 


0.4211 


0.(900 


0.5^(5 


c 


0.96^1 


O.I^CJ 


t.OlU 


c.^uoo 


O.t 


0.6^29 


0.26(1 


0. 506t) 


0.6750 


0. 5000 


f 


0.6766 


0.»}»? 


0.6^29 


0.6^29 


0.6^29 


0.0 


0.6^29 


0.6(16 


0.(000 


0.BI62 


a 


0.8929 


0.6?b6 


t.O/U 


0.(000 


6.26(1 


0.6^29 


0.0 


0.421t 


o.foai 


0.1162 


H 


0.1692 




0.1692 


0.^231 


0.(060 


0.6(16 


0.^211 


0.0 


0.(611 


0.5^55 


t 


o.^aii 


0.1061 


0.6290 


0.(000 


0.«?50 


0.(000 


0.1061 


0.(611 


0.0 


0.5909 


i 


0.6616 


0. ??27 


0.909t 


0.$45( 


0.(0t/0 


0.6162 


0.1162 


0.5459 


0.5909 


0.0 
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DISTANCE HATRIXt CIVIL ENGINEERING 





A 


B 


C 


D 


£ 


P 


G 


H 


I 


J 




A 


0.1) 


0.4000 


0*5600 


0*5600 


0.4600 


0.2600 


0.6400 


0«?91? 


0«619G 


0*521 




B 




0*0 




d.65?« 


0*7306 


0.4231 


0.5000 


0«6600 


0*5236 


0*563 




C 




0«3i46 


0.0 


0.60?? 


0.6^?? 


0.4231 


0«5?69 


d« 64410 


0«5?14 


0*563 




0 


0«S600 


0«653« 


0««0?? 


0«0 


0*2306 


0*6154 


0«3646 


0*6600 


0«6l90 


0*563 




£ 




O.?30« 


i>«00t? 


0.2306 


0«J 


0«f3«S 


0.4615 


0«6000 


0*6190 


0*563 




f 






0*4231 


0.6154 


0.5365 


0«0 


0*4615 


0.?20C 


0*5236 


0*451 




G 




O.SdOO 


0«5?6« 


0«3646 


0.4615 


0*4615 


0*0 


0.5600 


0*5236 


0.451 




H 


0W91? 


0«6dC0 


0«6400 


c«6aGa 


0«600C 


0.?200 


0*5630 


0«0 


0.9000 


0*B2( 


>% 


1 


0*6190 


0.5236 


0«5?U 


0*6190 


0«6H0 


0.5236 


0*5236 


0*9000 


0*0 


0*091 




i 


0«S2t? 


0.5B33 


0« 5633 


0.5633 


0.5633 


0.4563 


0.4563 


0*6261 


0*0952 


0*0 





103 



ERIC 



DISTANCE HATRIX: ELECTRICAL ENGINEERING 



A 


B 


c 


D 


B 


F 


G 


H 


I 


J 








.«34Ta 


r>«5652 


r*52iT 


d«6957 


i>«aia2 


n« 7500 


d*a5i>0 






o*434a 


o*e95f 


0*9110 


o«Ta2e 


0«tl696 


C«59C9 


1*3501) 


1*3500 






0*0 


0«6^5f 


0*B2f t 




0«86^6 


0«5909 


1*0000 


1*1000 


o«34ra 




0*6^57 


0*0 


0« 3r*^3 


o«434a 


0«60a7 


0«663£ 


o*a5co 


1*0500 


0*5652 




0*82(1 


0*3043 


0*0 


0*^Ta3 


0«5652 


1*0000 


o«aooo 


0*9000 


0*52X7 


0«T826 


o«6aaT 


o«^34a 


0*^703 


0*0 


0«3478 


0*7279 


0*6500 


0«£500 


0^6957 


d«a696 


C«a696 


D«60BT 


0*5652 


0«34Ta 


0«0 


0«86» 


0«6000 


0*£000 


o«aia2 


0*5909 


0« 59C9 


o«tiea6 


1*0000 


0« 7273 


0«a696 


0*0 


UIOOO 


1*2000 




t«350C 




o«a500 


o« aodo 


0«6 500 


0«6C00 


1*1000 


0*0 


0* 9000 


0*3500 


U3500 


UtOOd 


UOSJO 


C*9000 


0*6500 


0*6000 


1*2009 


0«9000 


0«0 
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DISTAHCB HATRIX: PSYCHOLOGY 





A 


B 


C 


D 


B 


F 


G 


H 


I 


J 


A 


0*0 


0.5417 


0.5417 


0. 541 7 


53Jt) 




w. DDD r 




0.6647 


d . 5000 


S 


0.5417 


0.0 


0*3333 


0.bbt7 


0*5417 


0.4lb7 


0.6250 


0.9565 


0*6667 


0*7273 


c 


0.5417 


0*3333 


0.0 


0.5«33 


0.5417 


0.tbb7 


0*5417 


0.7626 


0.6190 


0.6364 


0 


0*5417 


0.bbb7 


0.5S33 


0.0 


0.2917 


0.41b7 


0.45tt3 


0.60S7 


0.6667 


0.5909 


E 


0.^000 


0.5417 


0.5417 


0.2917 


0.0 


0.45«3 


0*5000 


0.5652 


0.6667 


0.5C00 


F 


0.5417 


0*41b7 


0.bbb7 


0.4tb7 


0.4503 


d.O 


0*7917 


0.«696 


0.7619 


0.7273 


Q 


0.bbb7 


3.6250 


0.5417 


0.45«3 


0*5000 


0.7917 


o.c 


0.6522 


0.7143 


0.6364 


H 


0*739 t 


0.95b5 


0.7«2b 


0.60«7 


0.5652 


O.Sb9b 


0.6522 


0.0 


0.4762 


0.3102 


1 


0.bbb7 


0.bbb7 


0.bl90 


0.bbb7 


0.bbb7 


0.7bl9 


0.7143 


0.4762 


0.0 


0.4266 


i 


0.5000 


0.7273 


0.b3b4 


0. 5909 


0*5000 


0.7273 


0*6 364 


0.31 $2 


0.42S6 


0.0 
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DISTANCE HATKIXt CHEMISTRY 



CDEP GHl J 



0*0 


o«asi9 


o«auo 


0«T43T 


0*5^26 


O«0U0 


0*740 7 


0*8000 


0«739t 


0*9909 


n«asi9 




0*3333 


a«6296 


0«T03T 


0*7037 


0«65t9 


0« 4800 


0«739t 


0*8t82 


0«SU8 


C«3333 


0*0 


o«e6eT 


0«666T 


3 « 6667 


C««Ull 


0«40C0 


0*5652 


C*7273 


0«HOT 


0«62S6 




0*0 


0« 3704 


o«5ta9 


0«6U6 


0*7200 


0«6997 


0*9909 


0«S926 


0«T03T 






0*3 


0*5926 


0*7407 


0*6800 


0*7391 


0*5909 


0«iU8 


0W03T 




o«stas 


0*5926 


0«0 


0*9926 


t«0000 


0*6922 


0*4949 




0«A$I9 


o«iiua 


o«aua 


o«noT 


0*5^26 


0*0 


U0800 


0*7826 


0*4364 


0*0000 


0«4i00 


0*4000 


0«T2C0 


o«6aoo 


t«0000 


t«oaoo 


C«0 


0*7826 


t*049S 


0«T39t 


0«T39t 


0*5652 


0«69ST 


0«T39t 


0*6522 


0«7a26 


0«7G26 


0*0 


0*3t82 


0*590^ 


0«dt82 


0W2T3 


0«99CS 


0*9909 


0«4545 


0«6364 


t«045S 


0*3t82 


0*0 



DISTANCE HATRIX: COMPUTER SCIENCE 





A 


It 


C 


D 


E 


F 


G 


H 


I 


J 


A 




0*66tl 


0*6250 


0«3f5d 


0.5000 


0*6250 


0*541? 


0*6667 


0*9091 


0*6000 


9 


0«666T 




0«45S3 


0*5417 


C*6667 


0.8?50 


0*?Cd3 


0*4286 


0*545$ 


0*7000 


C 




0«45a3 


0*0 


0*5833 


0.5417 


0*6667 


0.5a33 


0*6667 


0*6818 


0*6000 


H 


0*3 1^0 


0«54IT 


0«5i33 




6*2917 


0*5C00 


0*5C03 


0*6190 


0*6818 


0*5000 


c 


0*5600 




0*541? 


0*2917 


0*J 


0*4583 


0.3?50 


0*9524 


0*7273 


0*4500 


f 


0«£25U 


6*3 150 


0«666T 


0*5600 


0*4583 


0*0 


0.2500 


0.?143 


0*4091 


0.2000 




0*5417 


0*7003 


0*5833 


a.5coo 


0*3?50 


0*2500 


0*0 


0*7619 


0*5455 


0*2500 


H 


0«66&7 


0*42416 


0*666? 


C*6l90 


0*9524 


0*?I43 


0*?619 


0.0 


0*35f»O 


0*5556 


I 


0*9091 


:>.5455 


0«68U 


o*6aia 


0.72?3 


0*4091 


U*5455 


0*3500 


0*0 


0*2000 


i 


0«6000 


OWOOO 


0«6000 


0*5000 


0*4500 


0*2000 


0.2^00 


0*5556 


0*2000 


0*0 
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DISTANCB MATRIX: ALL DISCIPLINES (exclude HBA and Englleh) 





A 


B 


C 


D 


B 


F 


G 


H 




J 




0*0 


0*650^ 


0*^A29 


0*5203 


0*5tA5 


0*5610 


0.6505 


0*7652 


0*7 )i) 


0*6075 




0*6M>4 


0.0 


0*3171 


0*6371 


^#7^97 


0*6371 


0*7097 


0*629) 


0*7570 


0*0241 




0*6129 


0*3iTl 


0*0 


0*6C94 


0*6774 


d*6040 


0*6774 


0*6121 


0*6022 


0*7212 




0*)2d3 


^46371 


0*C694 


0*9 


0*2904 


0*5000 


0*5565 


0*6903 


0*7009 


0*6574 




0*51t) 


0*7097 


0*6774 


0*2904 


0*0 


0*5001 


0*532 3 


0*7500 


0*710) 


0*4010 




0*)610 


0*6371 


0*6041 


0*5000 


0*5001 


0*0 


0*4919 


0*0103 


0*5901 


0*5000 




Q*M$ 


0*7097 


0*6774 


0*5565 


0*5323 


0*4910 


0*0 


0*7045 


0*6)SS 


0*5105 


H 


0*7*52 


0*6293 


0*6121 


0*6903 


0*7500 


0*010) 


0*7045 


0*0 


o*7iia 


0*T9O5 


1 


C*T3i3 


0*7510 


0*6022 


0*7009 


0*7103 


0*5901 


0*6355 


0*7211 


0*0 


0*2492 


J 


0*6075 


0*6241 


0*T222 


0*6574 


0*6019 


0*5000 


0*5105 


0*7905 


0*2602 


0*0 



113 



o 

ERIC 



TOEFL Research Reports currently available . . . 

R«pcrt 1. The Performance of Nattve Speakers of Engit&hon the Tost of Engttsh as a Foretgn Langiiage 
Jotirt L. D. Clark. November 1977. 

Report 2. An £vatuatiOn of Affemattve item Formats for Testtng £ngfish as a FOretgn Language. 
Lewis W. Pike. June 1970. 

Report 3. The Performance of Non.NaUve Speakers of Engft&h on TOEFL and Verbaf Apmude Ter/s. 
Paul J Angelis. Sj>encef S. Swimon, and W^lllam R. Cowell. October 1979. 

Report 4, Art ExpJo^attOn of Speakmg Prottciency Measures in the TOEFL Context. John L D. Clark arnJ 
Spencer S. Swintor^. October 1979. 

Report 5. The Refauonshrp between Scores on the Graduate Management AdmrsstOn Test and the Test of 
Engttsh as a Foreign Language Donald Powers. D6cemt>er i960. 

Report 6. Factor Anatysts of the Test of Engftsh as a Foreign Language for Soverai Language Groups. 
Donald E. Powers and Spencer S. Swintort. De^^fmber 1960. 

Report 7. The Test of Spoken Engttsh as a Measure of Communicative AbiHty in Engiish-Medium tnstrvctionaf 
Settings. John L. D. Crark and Spencer S. Swinlon. December 1960. 

Report 8. EffeJs of item Discfosare on TOEFL Performance Gordon A. HaJe, Pauf J. AngeNS* and 
Lawrence A Thibodeau December 1960. 

Report 9. ttem Performance Across Nattve Language Groups on the T^st of English as a Foreign Language. 
Donald L. Atderman and Paul W. Holland. August 1961. 

Report 10. Language Proftctency as a Moderator Vanatyfe in Testing Academtc Aptitude. Donald L^ Alderman. 
November 1961. 

Report n* A Comparative Anafysisof TOEFL Exam/nee Characteristics^ 1977-1979. Kennelh M. Wrison. 
Juty 1932. 

Report 12. GMATand GRE Aptitude T^st Performance /n Reiation to Pnmaty Language and Scores on TOEFL 
Kenneth M. Wilson. July 1962. 

Report 1 3. The Test of Spoken English as a Measure of Communtcattve Abitity in the Heafth Professions: 
Vaiidatton ard Standard Setting. Donakf Po'A^ars and Charles W. Stanslield. January 1963. 

Report 14. A Manuat for Assasstng Language Growth in instructtOnai Settings. Spencer S. Swinlon. 
February 19&3. 

Report 15. Survey of Acadennc Wnttng Tasks Required of Graduate and Underg^duate Foretgn Students 
6rent 6ridgeman ^nd Sybil Carlson. Seplember 1963. 



If you wish additional information about TOEFL research or woufd like 
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